feat(typescript-beta): strengthen typescript grammar detection and in…

backlog/tasks/task-1 - Implement-TypeScript-schema-extension-with-FunctionDefinition-export-name-filtering.md

@@ -6,7 +6,7 @@ title: >-
 status: In Progress
 assignee: []
 created_date: '2026-02-25 06:16'
-updated_date: '2026-02-25 11:49'
+updated_date: '2026-02-28 07:01'
 labels:
   - typescript
   - schema
@@ -22,7 +22,13 @@ references:
     CallExpression-Confidence-Scoring-and-Validation-Agent.md
 documentation:
   - 'backlog://workflow/overview'
+  - 'https://nextjs.org/docs/app/building-your-application/routing/route-handlers'
+  - 'https://nextjs.org/docs/app/api-reference/file-conventions/route'
+  - 'https://tree-sitter.github.io/py-tree-sitter/classes/tree_sitter.Query.html'
+  - >-
+    https://tree-sitter.github.io/py-tree-sitter/classes/tree_sitter.QueryCursor.html
 priority: high
+ordinal: 1000
 ---
 
 ## Description
@@ -49,19 +55,121 @@ Out of scope
 - [ ] #4 Prisma-style call expression detection remains based on absolute imports + `CallExpression`, with validator handoff for ambiguous cases.
 - [ ] #5 No additional TypeScript concept enums are introduced beyond `FunctionDefinition` in this change.
 - [ ] #6 Implementation docs and examples remain aligned with the updated plan and validator assumptions.
+- [ ] #7 TypeScript language pipeline wires framework detection service so `.ts` files can emit `custom_features_list` detections for Next.js route handlers.
+- [ ] #8 Import-gated matching is enforced for TypeScript: exported HTTP handlers are only matched when `next/server` absolute-path imports are present.
+- [ ] #9 A canonical `framework-definitions/typescript/nextjs.json` definition is added and passes schema validation with existing v3 schema.
 <!-- AC:END -->
 
 ## Implementation Plan
 
 <!-- SECTION:PLAN:BEGIN -->
-1. Update shared contracts in commons and query-engine for FunctionDefinition and new construct_query regex keys.
-2. Extend canonical and public schema mirrors with FunctionDefinition and function/export regex fields.
-3. Relax absolute_paths schema pattern for TypeScript package/module forms while preserving terminal symbol requirement.
-4. Update flow-bridge loader to discover framework definition files across all language directories.
-5. Keep payload normalization compatible with FunctionDefinition values during ingestion.
-6. Update validation workflow commands and framework-definition contributor docs.
-7. Add targeted unit tests in commons/query-engine and loader-focused tests in flow-bridge.
-8. Run targeted schema validation, lint/type checks, and pytest suites.
+Objective
+Implement end-to-end TypeScript framework detection for Next.js route handlers by reusing the existing schema/DB contracts and mirroring the Python detection pipeline architecture.
+
+Scope Boundary
+- In scope: TypeScript `FunctionDefinition` detection for Next.js App Router route handlers in `.ts` files (e.g., `route.ts` exports like `GET`, `POST`).
+- Out of scope: validator-agent workflow integration (covered by TASK-1.2), TSX-specific route detection, additional TypeScript concepts beyond `FunctionDefinition`.
+
+Phase 1 - Framework Definition Catalog (ingestion input)
+1) Create `framework-definitions/typescript/nextjs.json`.
+2) Add library `nextjs` with feature `route_handler_export`:
+   - `concept`: `FunctionDefinition`
+   - `target_level`: `function`
+   - `startpoint`: `true`
+   - `construct_query.function_name_regex`: `^(GET|HEAD|POST|PUT|DELETE|PATCH|OPTIONS)$`
+   - `construct_query.export_name_regex`: `^(GET|HEAD|POST|PUT|DELETE|PATCH|OPTIONS)$`
+   - `absolute_paths`: import gates from `next/server` symbols (initially `next/server.NextRequest`, `next/server.NextResponse`)
+   - `base_confidence`: set explicitly for deterministic downstream confidence seeding.
+3) Keep schema unchanged unless validation reveals a gap; current v3 contract already includes needed fields.
+
+Phase 2 - TypeScript Source Context + Import Alias Model
+Algorithm (deterministic import extraction)
+1) Parse TypeScript source once using tree-sitter; keep a shared context object (`source_bytes`, `tree`, `root_node`).
+2) Walk `import_statement` nodes and extract:
+   - module source string (e.g., `next/server`)
+   - default import alias (`import x from 'm'` -> `m -> x`)
+   - namespace alias (`import * as ns from 'm'` -> `m -> ns`)
+   - named imports (`import { A, B as C } from 'm'` -> `m.A -> A`, `m.B -> C`)
+   - type-only named imports handled same as named imports for import gating.
+3) Build `import_aliases: dict[str, str]` in canonical form compatible with DB `absolute_paths` (e.g., `next/server.NextResponse -> NextResponse`, `next/server -> nextServer`).
+4) Deduplicate + preserve deterministic ordering for reproducible downstream matching/debug logs.
+
+Phase 3 - TypeScript Query Builder (`FunctionDefinition`)
+Algorithm (query generation)
+1) Add a TypeScript query template for exported function declarations:
+   - capture export statement (`@export_statement`)
+   - capture declaration node (`@function_definition`)
+   - capture function name (`@function_name`)
+   - capture export name (`@export_name`)
+2) Render predicates from `construct_query`:
+   - apply `#match? @function_name ...` when `function_name_regex` is present
+   - apply `#match? @export_name ...` when `export_name_regex` is present
+3) Cache compiled query by hash(feature definition payload) for performance parity with Python.
+
+Phase 4 - TypeScript Tree-Sitter Framework Detector
+Algorithm (feature matching)
+1) For each `FeatureSpec`, enforce import gating first:
+   - match if any `absolute_path` exists directly in `import_aliases`
+   - or module prefix strategy matches (symbol imported via module alias path).
+2) Concept dispatch:
+   - implement `Concept.FUNCTION_DEFINITION`
+   - explicitly skip unsupported concepts for TypeScript detector with debug logging.
+3) Execute query and create detections from `@function_definition` spans:
+   - `start_line`, `end_line`, `match_text`
+   - metadata: `concept='FunctionDefinition'`, `source='tree_sitter'`, `match_confidence=feature_spec.base_confidence`.
+4) Deduplicate identical span+feature rows in-memory before return.
+
+Phase 5 - TypeScript Framework Detection Service (DB-backed)
+Algorithm (service orchestration)
+1) Guard language == `typescript`; return empty otherwise.
+2) Build TypeScript source context.
+3) Expand import paths for DB lookup (module path and hierarchical prefixes where applicable).
+4) Query PostgreSQL using existing `get_framework_features_for_imports(session, 'typescript', imports)`.
+5) Pass `FeatureSpec` set to TypeScript detector and return detections.
+6) Error handling/logging should mirror Python service behavior.
+
+Phase 6 - Parser/Processor Wiring
+1) `CodeConfluenceCodebaseParser`:
+   - instantiate TypeScript framework detection service when codebase language is TypeScript.
+2) `TypeScriptLanguageProcessor`:
+   - extract imports (no longer `imports=None`)
+   - call injected framework detection service
+   - populate `custom_features_list` with detections
+   - keep existing data-model detection behavior intact.
+
+Phase 7 - Tests (unit + integration)
+1) Definition/loader tests:
+   - ensure `framework-definitions/typescript/nextjs.json` is loaded and preserves `FunctionDefinition` regex config.
+2) Source-context tests:
+   - validate alias extraction for default/named/aliased/namespace/type imports from `next/server`.
+3) Detector tests:
+   - positive: detects `export async function GET(...)` when `next/server` import exists
+   - negative: no detection without import gate
+   - regex filter: `GET` matches; non-HTTP export does not
+   - alias variant (`NextResponse as Resp`) still satisfies import gate.
+4) Processor tests:
+   - TypeScript processor returns `imports` and `custom_features_list` for Next.js route sample.
+5) Optional Postgres integration:
+   - DB-seeded TypeScript definitions produce detections via service path.
+
+Phase 8 - Validation + Quality Gates
+Execution order
+1) `uv run --group dev basedpyright src/`
+2) `uv run ruff check src/`
+3) `uv run --group dev check-jsonschema --schemafile framework-definitions/schema.json framework-definitions/*/*.json`
+4) `uv run --group test pytest` for targeted TypeScript framework-detection tests, then broader suite if needed.
+
+Key Risks + Mitigations
+- Risk: import alias edge cases in TypeScript grammar (type-only imports, aliases).
+  Mitigation: dedicated source-context tests for each import form.
+- Risk: false positives on exported helper functions.
+  Mitigation: strict HTTP-method regex + import gating on `next/server` symbols.
+- Risk: regression in existing TypeScript processor assumptions (`imports=None`).
+  Mitigation: update processor tests and keep backward-compatible optional imports field in `UnoplatFile`.
+
+Handoff Notes
+- Keep validator/agent-trigger logic untouched in this task.
+- If additional concepts are requested during implementation, pause and route as explicit scope change against TASK-1 acceptance criteria.
 <!-- SECTION:PLAN:END -->
 
 ## Implementation Notes
@@ -72,6 +180,12 @@ Set TASK-1 status to In Progress and prepared a concrete implementation sequence
 Implemented contract changes, schema updates, loader scanning updates, docs updates, and focused tests across flow-bridge, commons, and query-engine.
 
 Validation executed: check-jsonschema (framework-definitions/*/*.json), flow-bridge targeted pytest + ruff + basedpyright, commons targeted pytest, query-engine targeted pytest, and compileall checks.
+
+Detailed implementation flow was expanded on 2026-02-28 after repository audit. Key finding: schema/contracts already support `FunctionDefinition`; primary gap is missing TypeScript runtime pipeline (source-context/query-builder/detector/service wiring) and missing TypeScript framework definitions catalog file.
+
+Design decision: mirror Python detector architecture for parity and maintainability, but restrict TypeScript detector scope in this task to `FunctionDefinition` route handlers for Next.js App Router.
+
+Design decision: preserve confidence contract from TASK-1.1 by stamping `match_confidence` from `FeatureSpec.base_confidence` into detection metadata, allowing downstream validator flow in TASK-1.2 to remain unchanged.
 <!-- SECTION:NOTES:END -->
 
 ## Definition of Done

backlog/tasks/task-1.1 - Add-CallExpression-confidence-fields-to-schema-ingestion-docs-contracts-no-validator-integration.md

@@ -3,10 +3,10 @@ id: TASK-1.1
 title: >-
   Add CallExpression confidence fields to schema + ingestion docs/contracts (no
   validator integration)
-status: In Progress
+status: Done
 assignee: []
 created_date: '2026-02-26 05:53'
-updated_date: '2026-02-26 07:39'
+updated_date: '2026-02-28 06:04'
 labels:
   - schema
   - ingestion
@@ -40,6 +40,7 @@ documentation:
     unoplat-code-confluence-ingestion/code-confluence-flow-bridge/prisma-call-expression-simplification-failure-cases.md
 parent_task_id: TASK-1
 priority: high
+ordinal: 2000
 ---
 
 ## Description

backlog/tasks/task-1.2 - Integrate-low-confidence-CallExpression-validator-agent-into-query-engine-workflow.md

@@ -3,10 +3,10 @@ id: TASK-1.2
 title: >-
   Integrate low-confidence CallExpression validator agent into query-engine
   workflow
-status: In Progress
+status: Done
 assignee: []
 created_date: '2026-02-26 10:41'
-updated_date: '2026-02-27 12:43'
+updated_date: '2026-02-28 06:04'
 labels:
   - query-engine
   - validator
@@ -46,6 +46,7 @@ documentation:
   - 'backlog://workflow/overview'
 parent_task_id: TASK-1
 priority: high
+ordinal: 3000
 ---
 
 ## Description

backlog/tasks/task-1.2.1 - Fix-low-confidence-validation-regressions-in-candidate-hydration-and-mapper-gating.md

@@ -3,10 +3,10 @@ id: TASK-1.2.1
 title: >-
   Fix low-confidence validation regressions in candidate hydration and mapper
   gating
-status: In Progress
+status: Done
 assignee: []
 created_date: '2026-02-27 13:12'
-updated_date: '2026-02-28 05:09'
+updated_date: '2026-02-28 06:04'
 labels:
   - query-engine
   - validator
@@ -32,6 +32,7 @@ documentation:
     CallExpression-Confidence-Scoring-and-Validation-Agent.md
 parent_task_id: TASK-1.2
 priority: high
+ordinal: 4000
 ---
 
 ## Description

backlog/tasks/task-2 - Remaining-fixes-for-typescript-detection.md

@@ -0,0 +1,29 @@
+---
+id: TASK-2
+title: Remaining fixes for typescript detection
+status: To Do
+assignee: []
+created_date: '2026-02-28 06:59'
+labels: []
+dependencies: []
+---
+
+## Description
+
+<!-- SECTION:DESCRIPTION:BEGIN -->
+11. You missed updating TypeScript processor v1 contract comments/tests.
+- Gap: current file explicitly says framework/import extraction is deferred v1 in unoplat-code-confluence-ingestion/code-confluence-flow-bridge/src/code_confluence_flow_bridge/parser/language_processors/typescript_processor.py.
+- Fix: update docstring/tests in tests/parser/language_processors/test_typescript_processor.py that currently assert imports are intentionally omitted.
+12. You missed test impact from hardcoded framework counts.
+- Gap: several ingestion tests hardcode 4/7/11 counts in tests/integration/test_framework_definitions_ingestion.py.
+- Fix: if any test path starts loading all language dirs (not just python/), those assertions must become dynamic or updated.
+13. README/docs update is missing from execution steps.
+- Gap: framework-definitions/README.md still says TypeScript is “future framework definitions.”
+- Fix: update unoplat-code-confluence-ingestion/code-confluence-flow-bridge/framework-definitions/README.md once typescript/nextjs.json exists.
+14. Query-engine retrieval verification is missing as a test step.
+- Gap: plan validates ingestion/detection but not downstream retrieval behavior.
+- Fix: add at least one assertion path for db_get_all_framework_features_for_codebase(..., programming_language="typescript") in query-engine tests, especially if you keep non-standard feature key.
+15. Minor but important: route.ts file-convention gating is not represented.
+- Gap: your detector may match any exported GET/POST in any .ts importing next/server.
+- Fix: either explicitly accept this heuristic in task notes, or extend interface to pass file path and gate by filename in a follow-up.
+<!-- SECTION:DESCRIPTION:END -->

unoplat-code-confluence-ingestion/code-confluence-flow-bridge/framework-definitions/README.md

@@ -56,6 +56,39 @@ Detection is **import-gated** and **regex-based**:
 
 When `base_confidence` is below `0.70`, include disambiguation guidance in feature `notes` so downstream validation can apply framework-specific checks consistently.
 
+## Import Style For Detection Accuracy
+
+Framework matching depends on imports. Use explicit import style to improve recall and reduce false positives.
+
+- Python (enforced with Ruff):
+  - Prefer explicit imports (`from package import Symbol`) over wildcard imports.
+  - Keep absolute imports only (no relative imports).
+  - Relevant Ruff rules in this repo: `F403`, `F405`, `F406`, `TID`, `I`.
+
+- TypeScript/JavaScript (recommended with ESLint):
+  - Prefer named imports for framework entry modules (for example `next/server`).
+  - Avoid namespace imports for detection-critical modules.
+  - Avoid `require(...)` in ESM-first TypeScript codebases.
+  - Suggested rules: `import/no-namespace`, `@typescript-eslint/no-require-imports`, `import/no-commonjs`.
+
+Examples:
+
+```python
+# preferred
+from fastapi import APIRouter, FastAPI
+
+# avoid
+from fastapi import *
+```
+
+```typescript
+// preferred
+import { NextRequest, NextResponse } from "next/server"
+
+// avoid for detection-critical imports
+import * as nextServer from "next/server"
+```
+
 ## Validation
 
 ### Validate All Framework Definitions

unoplat-code-confluence-ingestion/code-confluence-flow-bridge/framework-definitions/typescript/nextjs.json

@@ -0,0 +1,22 @@
+{
+  "typescript": {
+    "nextjs": {
+      "docs_url": "https://nextjs.org/docs/app/building-your-application/routing/route-handlers",
+      "description": "Next.js App Router HTTP route handler exports in route.ts files.",
+      "features": {
+        "http_endpoint": {
+          "description": "App Router HTTP method handler: named export function (GET, POST, PUT, DELETE, PATCH, HEAD, OPTIONS). Requires next/server import to gate false positives.",
+          "absolute_paths": ["next/server.NextRequest", "next/server.NextResponse"],
+          "target_level": "function",
+          "concept": "FunctionDefinition",
+          "construct_query": {
+            "function_name_regex": "^(GET|HEAD|POST|PUT|DELETE|PATCH|OPTIONS)$",
+            "export_name_regex": "^(GET|HEAD|POST|PUT|DELETE|PATCH|OPTIONS)$"
+          },
+          "base_confidence": 0.95,
+          "startpoint": true
+        }
+      }
+    }
+  }
+}

unoplat-code-confluence-ingestion/code-confluence-flow-bridge/pyproject.toml

@@ -63,7 +63,7 @@ dev = [
 [tool.uv.sources]
 #NOTE: use it for local development to instantly see changes in the schema and its impact
 #unoplat-code-confluence-commons = { path = "../../unoplat-code-confluence-commons" }
-unoplat-code-confluence-commons = { git = "https://github.com/unoplat/unoplat-code-confluence.git", subdirectory = "unoplat-code-confluence-commons", rev = "unoplat-code-confluence-commons-v0.43.1" }
+unoplat-code-confluence-commons = { git = "https://github.com/unoplat/unoplat-code-confluence.git", subdirectory = "unoplat-code-confluence-commons", rev = "unoplat-code-confluence-commons-v0.44.0" }
 
 [tool.basedpyright]
 typeCheckingMode = "strict"

unoplat-code-confluence-ingestion/code-confluence-flow-bridge/src/code_confluence_flow_bridge/engine/framework_detection_service.py

@@ -2,14 +2,25 @@
 Abstract base class for language-specific framework detection services.
 """
 
+from __future__ import annotations
+
 from abc import ABC, abstractmethod
-from typing import List, Optional
+from typing import TYPE_CHECKING, List, Optional
 
 from unoplat_code_confluence_commons.base_models import (
     Detection,
     PythonStructuralSignature,
+    TypeScriptStructuralSignature,
 )
 
+if TYPE_CHECKING:
+    from src.code_confluence_flow_bridge.engine.programming_language.python.python_source_context import (
+        PythonSourceContext,
+    )
+    from src.code_confluence_flow_bridge.engine.programming_language.typescript.typescript_source_context import (
+        TypeScriptSourceContext,
+    )
+
 
 class FrameworkDetectionService(ABC):
     """Abstract base for language-specific framework detection services."""
@@ -19,8 +30,9 @@ async def detect_features(
         self,
         source_code: Optional[str],
         imports: List[str],
-        structural_signature: PythonStructuralSignature | None,
+        structural_signature: PythonStructuralSignature | TypeScriptStructuralSignature | None,
         programming_language: str,
+        source_context: PythonSourceContext | TypeScriptSourceContext | None = None,
     ) -> List[Detection]:
         """
         Detect framework features in source code using imports and structural signature.
@@ -29,7 +41,8 @@ async def detect_features(
             source_code: Source code to analyze
             imports: List of imports in the file
             structural_signature: Structural signature of the file (optional)
-            programming_language: Programming language (e.g., "python")
+            programming_language: Programming language (e.g., "python", "typescript")
+            source_context: Pre-parsed source context to avoid double-parsing (optional)
 
         Returns:
             List of Detection objects for framework features found