Skip to content

Phase 1.1: QA Checklist module path picker — browse the indexed tree instead of typing a path #17

Description

@zulfikar-ditya

The problem

POST /checklist-modules scopes a QA Checklist module to source_path, a repository-relative path. Today it is a free-typed text field with no browsing and no validation against the indexed repository.

A typo'd path returns 201. Nothing goes wrong until later and elsewhere: the background generation cannot match anything under it and fails with MODULE_PATH_NOT_INDEXED, minutes after the user left the form.

That is tolerable for someone who already knows the tree. It is a wall for someone who does not — and "someone who does not" is exactly the phase-1.1 audience. Phase 1 was built for the team that stood the instance up and already knows its own repository's layout; phase 1.1 is the same feature set, same access model, opened to a QA lead evaluating the tool, a contractor, anyone whose first contact with the repository is through AskRepo itself. Their first action on the product is to type a path they cannot know, and the only feedback is a background job that fails.

To be explicit about what this is not: this does not touch the access resolver (backend/app/core/access.py). Nothing about who may see what changes. It is a generalization item, not an early phase-2 item.

Proposed solution

A path picker backed by a new read of the project's own indexed file_path values, which already sit in the vector store.

  • A distinct scroll, not a search. VectorStore.scroll (backend/app/ingestion/vector_store.py:158) already exists for the checklist generator. This read collects distinct file_path payload values for the project — no vector search, no chat-model call, no embedding call. The cost of this read is the whole reason it is a scroll.
  • Generation-scoped and cached per (project_id, active_generation), invalidated on reindex the same way every other generation-scoped read is. A reindex is a generation swap (.claude/rules/ingestion.md), so the cache key falls out of the model that already exists rather than needing new invalidation machinery.
  • The collection comes from the project rowproject.embedding_collection, verbatim, never recomputed from current settings (.claude/rules/rag.md).
  • Validate at creation. POST /checklist-modules rejects a source_path matching nothing indexed, so the error arrives while the user is looking at the form instead of in a worker log.
  • Frontend: browse or search the real tree during module creation instead of guessing at it.

Acceptance criteria:

  • A new read returns the distinct indexed paths for a project's active generation, scoped through resolve_project_scope like every other project read.
  • Creating a module with an unmatched source_path fails at POST /checklist-modules, not at generate time.
  • The module-create UI lets a user browse/search paths without typing one from memory.
  • The cache is invalidated by a reindex (generation bump), with a test.
  • A test asserts no vector search and no model call happen on this path.
  • MODULE_PATH_NOT_INDEXED still exists for the genuine race — path indexed at create time, gone after a reindex. It is simply no longer the first place a typo is caught.

Docs that move in the same change: docs/PRD.md §4.3 and §2.1, backend/README.md route table, docs/rag.md (a new generation-scoped read), CHANGELOG.md under ## [Unreleased].

Alternatives you considered

Do nothing. Viable while the only users are the team that wrote the code. It becomes the first thing that stops an evaluator, and it fails in the worst possible way — silently, with a delay, in a place the user is not looking.

Validate at creation but keep the free-typed field. Cheaper, and it turns a silent late failure into an immediate 422. But it still requires the user to produce a correct path from memory before they get any feedback at all, which is the actual barrier. Worth shipping as the first half if the picker is deferred.

Serve the tree from a fresh git ls-tree instead of the index. Rejected: the cloned working copy is deleted after indexing (/data/repos is scratch), so this would mean re-cloning to populate a dropdown. The vector store already holds every indexed path, and reading it there means the picker offers exactly what generation can actually match — the two cannot disagree.

Top-k search over paths. Rejected for the same reason the checklist generator scrolls rather than searches: top-k cannot report what it left out, and a picker that silently omits a directory is worse than no picker.

Area

QA Checklist

Which phase does this belong to?

Phase 1.1 (deferred generalization — docs/PRD.md §2.1)

Before submitting

  • I checked docs/PRD.md and this isn't already planned or listed as a non-goal.
  • I searched existing issues for this.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestphase-1.1Deferred generalization for users who have not seen the repository (PRD 2.1)

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions