feat(log): add append-only activity journal (log.md)

[alvins82]

What

Adds the log.md journal from Karpathy's llm-wiki gist: an append-only record of what happened and when, written automatically by ingest, compile, query, and lint.

Each entry is a grep-able heading followed by a markdown bullet body:

## [2026-06-05T09:14:02Z] ingest | Attention Is All You Need
- Source: https://arxiv.org/abs/1706.03762
- Saved: sources/attention-is-all-you-need.md
- Chars: 38,214

## [2026-06-05T09:15:30Z] compile | 1 source(s) → 6 page(s)
- Sources: attention-is-all-you-need.md
- Created: [[self-attention]], [[multi-head-attention]], [[transformer]]
- Updated: [[positional-encoding]]

## [2026-06-05T09:16:11Z] query | What is multi-head attention?
- Pages: [[multi-head-attention]], [[self-attention]]

Why

The gist treats log.md as a companion to index.md: where the index organizes content for discovery, the log tracks temporal progression. The project had no such journal.

Design notes

• Format: ## [YYYY-MM-DDThh:mm:ssZ] operation | description (ISO 8601 UTC) + a - detail bullet body. Only headings start with ## [, so the gist's recipe grep "^## \[" log.md | tail -5 still returns the most recent operations even with bodies.
• Per-operation detail:
  • ingest — Source (input), Saved (output file), Chars
  • compile — Sources consumed, Created vs Updated pages (split via a pre-generation on-disk snapshot), Deleted sources count
  • query — cited Pages
  • lint — error/warning/info counts + Flagged pages
• Placement: logging lives in the core functions (ingestSource, runCompilePipeline, generateAnswer, lint), so both the CLI and the MCP server are covered. It writes to log.md at the project root (not under wiki/) because ingest runs before any wiki/ exists.
• Resilient: a log write failure warns but never throws — recording an event must not break the operation it records.
• log.md is gitignored alongside /wiki/ and /sources/.
• Page-link lists are capped (20) with a (+N more) suffix.

Out of scope

Token/cost was intentionally deferred to a follow-up (pairs naturally with the Claude Agent SDK provider, which reports usage/cost natively).

Testing

• New test/activity-log.test.ts covers heading format, ISO timestamp, bullet body, wikilink/list truncation, append-only accumulation, and the never-throws guarantee.
• npx tsc --noEmit, npm run build, npm test (1216 passed), and fallow (no dead code/duplication, 0 above threshold, maintainability 91.5) all pass.
• Verified end-to-end against a real provider: ingest → compile → query → lint all journal correctly, including the Created/Updated split across two compiles.

🤖 Generated with Claude Code

[ethanj]

Thanks for putting this together. The core idea is useful, and the implementation is cleanly documented. I especially like that the log format keeps the gist’s grep "^## \\[" log.md | tail -5 workflow intact while still allowing richer detail lines.

I’d like to request a few changes before merging:

1. lint_wiki now mutates state, which conflicts with the MCP server contract. src/mcp/server.ts currently tells agents that read_page, lint_wiki, wiki_status, and fast unrecorded eval “do not mutate state,” but lint() now appends to log.md on every run, including clean 0/0/0 passes. Could we either avoid journaling lint from that read-only path, or deliberately update the contract and add a regression test showing lint_wiki is expected to write log.md?

2. The compile created/updated split is re-derived from a bare-slug directory scan. listExistingPageSlugs() merges wiki/concepts/*.md and wiki/queries/*.md into one Set<string>, so an existing query page and a newly-created concept with the same slug can be misclassified as “Updated.” It would be better to thread the created/updated result from the exact page write path, or at least track namespaced IDs like concepts/foo and queries/foo.

3. The actual journaling call sites need integration coverage. test/activity-log.test.ts covers the formatter and append helper, but not the real compile/lint/query/ingest paths. A small CLI-level test would make sure the feature keeps working through the user-facing surface and would catch issues like lint mutating unexpectedly or compile misreporting created vs updated pages.

A few smaller non-blocking notes:

• generateAnswer() logs the query before the answer is generated, so a later LLM failure still records a query entry. That may be fine if the journal records attempts, but it is worth making that intentional because compile logs after successful finalization.
• ingestSource() writes the log using process.cwd(), while compile/query/lint pass an explicit root. It works for the current CLI/MCP path, but passing root explicitly would make the invariant less fragile.
• produced = [...pages, ...seedSlugs] is not deduped, so a collision could inflate the compile page count.

Good feature overall. Once the lint contract and compile classification are tightened, I think this will fit the repo well.

[alvins82]

Thanks for the thorough review — all three blocking items addressed in f382606, plus two of the non-blocking notes. (Also rebased onto the latest branch head so #87's dep upgrade is included.)

1. lint_wiki read-only contract ✅
Moved the lint journaling out of the core lint() function and into the CLI command (src/commands/lint.ts). The MCP lint_wiki tool calls lint() directly, so it no longer writes log.md; only llmwiki lint (CLI) journals. The contract in src/mcp/server.ts stands as-is. Added a regression test asserting core lint() does not write log.md alongside one asserting the CLI does.

2. Created/Updated misclassification ✅
listExistingPageIds() now namespaces the snapshot by directory (wiki/concepts/foo vs wiki/queries/foo) instead of merging bare slugs into one set. Since compile only ever writes concept-namespace pages (concept + seed pages both land under wiki/concepts/), existence is checked against concepts/<slug> — so an existing query page and a same-slug new concept are no longer conflated as "Updated."

3. Integration coverage for the call sites ✅
New test/activity-log-integration.test.ts exercises the real surfaces:

• CLI llmwiki lint writes a lint entry (via the runCLI subprocess fixture)
• core lint() stays read-only (no log.md)
• CLI llmwiki ingest <file> writes an ingest entry with Saved/Chars
• compile logs Created on first compile and Updated on recompile (stubbed provider, two passes)

Non-blocking notes:

• query logs before generation ✅ — moved the query journal to after the answer is produced, matching compile (which logs after finalization), so a mid-flight LLM failure records nothing.
• produced not deduped ✅ — deduped the produced slugs before counting/splitting, so a concept/seed slug collision can't inflate the page count.
• ingestSource() uses process.cwd() — left as-is intentionally: the whole ingest path is cwd-relative (saveSource writes sources/ relative to cwd too), so process.cwd() for the journal is consistent. Threading an explicit root only into the log call would be inconsistent without also refactoring saveSource; happy to do that as a follow-up if you'd prefer.

tsc, build, npm test (1245 passed), and fallow (clean) all green.

[alvins82]

Follow-up (9f7e524): decided to drop lint from the journal entirely rather than journal it from the CLI. log.md now records only ingest, compile, and query. lint is a read-only check (and lint_wiki is documented non-mutating), so it leaves no trace anywhere — which most cleanly resolves the read-only-contract concern from point 1. The integration test now asserts llmwiki lint writes no log.md. Docs updated. All checks green.

[ethanj]

Thanks for the thorough follow-up. I re-reviewed the latest changes and this is in good shape now.

The main concerns from the earlier pass are addressed: lint_wiki preserves the no-mutation contract again, the created/updated split is now namespaced so concept/query slug collisions do not contaminate the journal, and the real call sites are covered through integration tests instead of only testing the formatter/helper layer.

The end-to-end coverage is the important part here. Pinning lint as read-only, ingest details, and first-compile-created / second-compile-updated behavior gives this feature the right protection going forward.

Approving. Nice work on the journal.