Skip to content

Commit 072b16c

Browse files
debpalashclaude
andcommitted
test(integration): fix wing scope, un-pin schema version and tool count
With the SIGILL fixed, CI finally reached the integration step — which had been unreachable for so long that all six of its failures were rot in the test, not in memxt. Verified locally: the product is fine; only the assertions were wrong. * ranking (4 failures): step 2 mines into wing "testwing", but rank() ran a bare `search`, which scopes to the DEFAULT wing (derived from the git repo → "memxt") and is empty. Every ranking check compared against '' — a wing mismatch masquerading as a ranking regression. Search --wing testwing. (Same class as f48ea5b "fix release smoke search wing scope"; that fixed the release smoke and left this one.) * schema: asserted PRAGMA user_version == 1. The schema is legitimately at v5, so the literal made the check fail on every migration. Assert it is stamped (> 0). * MCP tools: asserted exactly 4 tools; there are 8. An exact count tells you nothing about whether the server works and breaks whenever a tool is added. Assert the contract instead — the core tools an agent depends on are exposed. Local: integration 14 passed, 0 failed. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1 parent 86d8e0e commit 072b16c

1 file changed

Lines changed: 27 additions & 5 deletions

File tree

scripts/integration-test.sh

Lines changed: 27 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -28,9 +28,15 @@ echo " model: $MEMXT_MODEL"
2828

2929
"$BIN" init >/dev/null 2>&1
3030

31-
# 1) schema version is stamped
31+
# 1) schema version is stamped. Assert it's SET, not a specific number — the
32+
# schema legitimately evolves (it is v5 now), and pinning the literal just makes
33+
# this fail on every migration, which is what it did.
3234
ver=$(sqlite3 "$MEMXT_DB" "PRAGMA user_version;" 2>/dev/null || echo "")
33-
[ "$ver" = "1" ] && ok "schema version stamped (=1)" || bad "schema version not stamped (got '$ver')"
35+
if [ -n "$ver" ] && [ "$ver" -gt 0 ] 2>/dev/null; then
36+
ok "schema version stamped (=$ver)"
37+
else
38+
bad "schema version not stamped (got '$ver')"
39+
fi
3440

3541
# 2) directory mining ingests multiple files
3642
CORPUS="$WORK/corpus"; mkdir -p "$CORPUS"
@@ -41,8 +47,12 @@ printf 'The French Revolution began in 1789 and led to the rise of Napoleon Bona
4147
mine_out=$("$BIN" mine "$CORPUS" testwing 2>&1)
4248
have "$mine_out" "Drawers created: 4" && ok "directory mine ingested 4 files" || bad "directory mine wrong drawer count: $(echo "$mine_out" | grep Drawers)"
4349

44-
# 3) semantic ranking: paraphrase queries (no shared keywords) hit the right file
45-
rank() { "$BIN" search "$1" 2>&1 | grep -m1 "Source:" | sed 's/.*Source: //'; }
50+
# 3) semantic ranking: paraphrase queries (no shared keywords) hit the right file.
51+
# MUST search --wing testwing: step 2 mines into "testwing", but a bare `search`
52+
# scopes to the default wing (derived from the git repo → "memxt"), which is
53+
# empty here. Without this the ranking assertions compared against '' and failed
54+
# for a wing mismatch, not a ranking regression.
55+
rank() { "$BIN" search "$1" --wing testwing 2>&1 | grep -m1 "Source:" | sed 's/.*Source: //'; }
4656
check_rank() { local got; got=$(rank "$1"); [ "$got" = "$2" ] && ok "rank: '$1' -> $2" || bad "rank: '$1' -> got '$got', want $2"; }
4757
check_rank "how do biological cells produce energy" "biology.txt"
4858
check_rank "brewing a great cup of coffee" "coffee.txt"
@@ -58,7 +68,19 @@ mcp_out=$(printf '%s\n' \
5868
| "$BIN" mcp 2>/dev/null)
5969
have "$mcp_out" '"protocolVersion":"2025-11-25"' && ok "MCP initialize echoes protocol version" || bad "MCP initialize bad"
6070
# tools/list is one JSON line, so count occurrences (grep -o), not matching lines.
61-
[ "$(echo "$mcp_out" | grep -o 'inputSchema' | wc -l | tr -d ' ')" = "4" ] && ok "MCP tools/list exposes 4 tools" || bad "MCP tools/list wrong tool count"
71+
# Assert the CONTRACT (the core tools an agent depends on are exposed) rather than
72+
# an exact count: the old `= 4` broke the moment tools were added (there are 8),
73+
# which tells you nothing about whether the server still works.
74+
missing=""
75+
for t in memory_search memory_store memory_wake_up memory_get memory_forget; do
76+
echo "$mcp_out" | grep -q "\"name\":\"$t\"" || missing="$missing $t"
77+
done
78+
n_tools=$(echo "$mcp_out" | grep -o 'inputSchema' | wc -l | tr -d ' ')
79+
if [ -z "$missing" ] && [ "$n_tools" -ge 5 ]; then
80+
ok "MCP tools/list exposes the core tools ($n_tools total)"
81+
else
82+
bad "MCP tools/list missing:$missing (count=$n_tools)"
83+
fi
6284
have "$mcp_out" "QUASAR-77" && ok "MCP store -> recall round-trip" || bad "MCP store/recall failed"
6385

6486
# 5) hooks: SessionStart injects context; PreCompact saves the transcript tail

0 commit comments

Comments
 (0)