Skip to content

feat(lsp): surface traceability gaps in family gate #223

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
newtontech merged 1 commit into from
Jun 15, 2026
Merged

feat(lsp): surface traceability gaps in family gate #223

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
188 changes: 120 additions & 68 deletions docs/LSP_COMPATIBILITY.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,54 +1,117 @@
# LSP Compatibility Matrix

This matrix is the OpenQC-facing contract for the newtontech LSP family. The source of truth is `src/lsp/registry.ts`; `package.json`, language configurations, TextMate grammars, and this document must stay aligned with that registry.

Last updated: 2026-06-15. Latest support is tracked against each repository default branch unless a release tag is listed below. OpenQC does not pin old server binaries; it launches the configured executable or sibling local repository, so users can run the newest server available in their environment.

## Quick Reference

| LSP Server | Language ID | Default Branch | Latest Release/Local Version | Domain | File Types | Stability | OpenQC Integration |
|------------|-------------|----------------|------------------------------|--------|------------|-----------|--------------------|
| [newtontech/abacus-lsp](https://github.com/newtontech/abacus-lsp) | `abacus` | `main` | 0.1.0 | ABACUS | `INPUT`, `STRU`, `KPT` | Experimental | Syntax, file detection, direct LSP startup |
| [newtontech/abinit-lsp](https://github.com/newtontech/abinit-lsp) | `abinit` | `main` | 0.1.0 | ABINIT | `.abi`, `.abinit` | Experimental | Syntax, file detection, direct LSP startup |
| [newtontech/cif-lsp](https://github.com/newtontech/cif-lsp) | `cif` | `master` | v1.0.2 | CIF | `.cif` | Experimental | Syntax, file detection, direct LSP startup |
| [newtontech/cp2k-lsp-enhanced](https://github.com/newtontech/cp2k-lsp-enhanced) | `cp2k` | `develop` | v0.9.1 | CP2K | `.inp` | Experimental | Syntax, file detection, direct LSP startup |
| [newtontech/VASP-LSP](https://github.com/newtontech/VASP-LSP) | `vasp` | `main` | 0.4.4 | VASP | `INCAR`, `POSCAR`, `KPOINTS`, `POTCAR`, `CONTCAR`, `OSZICAR`, `OUTCAR`, `vasprun.xml` | Stable | Syntax, file detection, direct LSP startup |
| [newtontech/gaussian-lsp](https://github.com/newtontech/gaussian-lsp) | `gaussian` | `main` | 0.2.11 | Gaussian | `.gjf`, `.com` | Stable | Syntax, file detection, direct LSP startup |
| [newtontech/orca-lsp](https://github.com/newtontech/orca-lsp) | `orca` | `main` | 0.5.5 | ORCA | `.inp` | Stable | Syntax, file detection, direct LSP startup |
| [newtontech/qe-lsp](https://github.com/newtontech/qe-lsp) | `qe` | `main` | 0.1.0 | Quantum ESPRESSO | `.in`, `.pw.in`, `.relax.in`, `.vc-relax.in`, `.scf.in`, `.nscf.in`, `.bands.in`, `.ph.in`, `.dos.in` | Stable | Syntax, file detection, direct LSP startup |
| [newtontech/gamess-lsp](https://github.com/newtontech/gamess-lsp) | `gamess` | `main` | 0.1.0 | GAMESS | `.inp` | Stable | Syntax, file detection, direct LSP startup |
| [newtontech/nwchem-lsp](https://github.com/newtontech/nwchem-lsp) | `nwchem` | `main` | v0.3.0 / local 0.5.0 | NWChem | `.nw`, `.nwinp` | Experimental | Syntax, file detection, direct LSP startup |
| [newtontech/gpumd-lsp](https://github.com/newtontech/gpumd-lsp) | `gpumd` | `main` | 0.1.0 | GPUMD | `run.in`, `nep.in` | Experimental | Syntax, file detection, direct LSP startup |
| [newtontech/gromacs-lsp](https://github.com/newtontech/gromacs-lsp) | `gromacs` | `main` | v0.0.2 | GROMACS | `.top`, `.itp`, `.mdp`, `.gro` | Experimental | Syntax, file detection, direct LSP startup |
| [newtontech/lammps-lsp](https://github.com/newtontech/lammps-lsp) | `lammps` | `master` | 0.1.0-pre-release-3 | LAMMPS | `.lmp`, `.lammps`, `.lmps`, `in.lammps` | Experimental | Syntax, file detection, direct LSP startup |
| [newtontech/mlip-lsp](https://github.com/newtontech/mlip-lsp) | `mlip` | `main` | 0.1.0 | MLIP | `.mlip.json`, `.mlip.yaml`, `.mlip.yml`, `mlip.json`, `mlip.yaml`, `mlip.yml` | Experimental | Syntax, file detection, direct LSP startup |
| [newtontech/pyatb-lsp](https://github.com/newtontech/pyatb-lsp) | `pyatb` | `main` | 0.1.0 | PyATB | `.pyatb.py`, `run_pyatb.py` | Experimental | Syntax, file detection, direct LSP startup |
| [newtontech/pyscf-lsp](https://github.com/newtontech/pyscf-lsp) | `pyscf` | `main` | 0.1.0 | PySCF | `.pyscf.py`, `run_pyscf.py` | Experimental | Syntax, file detection, direct LSP startup |
| [newtontech/dpgen-lsp](https://github.com/newtontech/dpgen-lsp) | `dpgen` | `main` | 0.1.0 | DP-GEN | `param.json`, `machine.json` | Experimental | Syntax, file detection, direct LSP startup |

## Diagnostic Engine v1 Readiness

Every standalone LSP exposes the same agent CLI operation set: `check`, `context`, `complete`, `hover`, `symbols`, and `fix`. Optional provider depth can vary by backend, but unavailable provider data must be reported explicitly in JSON instead of returning reserved placeholders.

| LSP Server | Diagnostic Engine | Agent CLI | Agent Operations | Help Smoke | Closed Loop |
|------------|-------------------|-----------|------------------|------------|-------------|
| `abacus-lsp` | v1 | `abacus-lsp-tool` | `check`, `context`, `complete`, `hover`, `symbols`, `fix` | Passing | Partial |
| `abinit-lsp` | v1 | `abinit-lsp-tool` | `check`, `context`, `complete`, `hover`, `symbols`, `fix` | Passing | Partial |
| `cif-lsp` | v1 | `cif-lsp-tool` | `check`, `context`, `complete`, `hover`, `symbols`, `fix` | Pending | Planned |
| `cp2k-lsp-enhanced` | v1 | `cp2k-lsp-tool` | `check`, `context`, `complete`, `hover`, `symbols`, `fix` | Pending | Partial |
| `gamess-lsp` | v1 | `gamess-lsp-tool` | `check`, `context`, `complete`, `hover`, `symbols`, `fix` | Pending | Planned |
| `gaussian-lsp` | v1 | `gaussian-lsp-tool` | `check`, `context`, `complete`, `hover`, `symbols`, `fix` | Passing | Partial |
| `gpumd-lsp` | v1 | `gpumd-lsp-tool` | `check`, `context`, `complete`, `hover`, `symbols`, `fix` | Pending | Planned |
| `gromacs-lsp` | v1 | `gromacs-lsp-tool` | `check`, `context`, `complete`, `hover`, `symbols`, `fix` | Pending | Planned |
| `lammps-lsp` | v1 | `lammps-lsp-tool` | `check`, `context`, `complete`, `hover`, `symbols`, `fix` | Pending | Partial |
| `mlip-lsp` | v1 | `mlip-lsp-tool` | `check`, `context`, `complete`, `hover`, `symbols`, `fix` | Pending | Planned |
| `nwchem-lsp` | v1 | `nwchem-lsp-tool` | `check`, `context`, `complete`, `hover`, `symbols`, `fix` | Pending | Planned |
| `orca-lsp` | v1 | `orca-lsp-tool` | `check`, `context`, `complete`, `hover`, `symbols`, `fix` | Passing | Partial |
| `pyatb-lsp` | v1 | `pyatb-lsp-tool` | `check`, `context`, `complete`, `hover`, `symbols`, `fix` | Pending | Planned |
| `pyscf-lsp` | v1 | `pyscf-lsp-tool` | `check`, `context`, `complete`, `hover`, `symbols`, `fix` | Passing | Partial |
| `dpgen-lsp` | v1 | `dpgen-lsp-tool` | `check`, `context`, `complete`, `hover`, `symbols`, `fix` | Pending | Planned |
| `qe-lsp` | v1 | `qe-lsp-tool` | `check`, `context`, `complete`, `hover`, `symbols`, `fix` | Pending | Partial |
| `vasp-lsp` | v1 | `vasp-lsp-tool` | `check`, `context`, `complete`, `hover`, `symbols`, `fix` | Pending | Partial |
This document is generated from `src/lsp/registry.ts` and the LSP family gate report.
Regenerate it with `npm run lsp:generate-compatibility-doc` after registry, backend, or gate changes.

## Release Gate Summary

| Metric | Value |
|--------|-------|
| Backends | 17 |
| Passing | 17 |
| Blocking gaps | 0 |
| Warnings | 18 |
| Graduation score | 78/100 |

## Backend Matrix

| Backend | Language | Branch | File Types | Registry Stability | Gate Maturity | Release Evidence | Manifest | Provenance | Fixtures | Smoke | Diagnostics | Traceability |
|---------|----------|--------|------------|--------------------|---------------|------------------|----------|------------|----------|-------|-------------|--------------|
| [newtontech/abacus-lsp](https://github.com/newtontech/abacus-lsp) | `abacus` | `main` | `INPUT`, `STRU`, `KPT` | experimental | stable | `v0.1.0`<br>VERSION `0.1.0`<br>HEAD `1e5d05f91b3f` | pass | pass | pass | pass | pass | missing |
| [newtontech/abinit-lsp](https://github.com/newtontech/abinit-lsp) | `abinit` | `main` | `.abi`, `.abinit` | experimental | stable | `v0.1.0`<br>VERSION `0.1.0`<br>HEAD `7612aeda243b` | pass | pass | pass | pass | pass | missing |
| [newtontech/cif-lsp](https://github.com/newtontech/cif-lsp) | `cif` | `master` | `.cif` | experimental | stable | `v1.0.2`<br>HEAD `b3784d69cece` | pass | pass | pass | pass | pass | missing |
| [newtontech/cp2k-lsp-enhanced](https://github.com/newtontech/cp2k-lsp-enhanced) | `cp2k` | `develop` | `.inp` | experimental | stable | `v0.9.1`<br>HEAD `c38db0a58344` | pass | pass | pass | pass | pass | missing |
| [newtontech/VASP-LSP](https://github.com/newtontech/VASP-LSP) | `vasp` | `main` | `INCAR`, `POSCAR`, `KPOINTS`, `POTCAR`, `CONTCAR`, `OSZICAR`, `OUTCAR`, `vasprun.xml` | stable | stable | `v0.4.4`<br>VERSION `0.4.4`<br>HEAD `587fd094f7a7` | pass | pass | pass | pass | pass | missing |
| [newtontech/gaussian-lsp](https://github.com/newtontech/gaussian-lsp) | `gaussian` | `main` | `.gjf`, `.com` | stable | stable | `v0.2.11`<br>VERSION `0.2.11`<br>HEAD `e886a6553cd9` | pass | pass | pass | pass | pass | missing |
| [newtontech/orca-lsp](https://github.com/newtontech/orca-lsp) | `orca` | `main` | `.inp` | stable | stable | `v0.5.5`<br>VERSION `0.5.5`<br>HEAD `ecd4cabdba75` | pass | pass | pass | pass | pass | missing |
| [newtontech/qe-lsp](https://github.com/newtontech/qe-lsp) | `qe` | `main` | `.in`, `.pw.in`, `.relax.in`, `.vc-relax.in`, `.scf.in`, `.nscf.in`, `.bands.in`, `.ph.in`, `.dos.in` | stable | stable | `v0.1.0`<br>VERSION `0.1.0`<br>HEAD `ab0c4ac6cee1` | pass | pass | pass | pass | pass | missing |
| [newtontech/gamess-lsp](https://github.com/newtontech/gamess-lsp) | `gamess` | `main` | `.inp` | stable | stable | `v0.1.0`<br>VERSION `0.1.0`<br>HEAD `1dd85afa3d85` | pass | pass | pass | pass | pass | missing |
| [newtontech/nwchem-lsp](https://github.com/newtontech/nwchem-lsp) | `nwchem` | `main` | `.nw`, `.nwinp` | experimental | stable | `v0.3.0`<br>HEAD `a9109413251e` | pass | pass | pass | pass | pass | missing |
| [newtontech/gpumd-lsp](https://github.com/newtontech/gpumd-lsp) | `gpumd` | `main` | `run.in`, `nep.in` | experimental | stable | `v0.1.0`<br>VERSION `0.1.0`<br>HEAD `882a582a23dd` | pass | pass | pass | pass | pass | missing |
| [newtontech/gromacs-lsp](https://github.com/newtontech/gromacs-lsp) | `gromacs` | `main` | `.top`, `.itp`, `.mdp`, `.gro` | experimental | stable | `v0.0.3`<br>VERSION `0.0.3`<br>HEAD `3074ac19d5a7` | pass | pass | pass | pass | pass | missing |
| [newtontech/lammps-lsp](https://github.com/newtontech/lammps-lsp) | `lammps` | `master` | `.lmp`, `.lammps`, `.lmps`, `in.lammps` | experimental | stable | `0.1.0-pre-release-3`<br>HEAD `8d77f45d8229` | pass | pass | pass | pass | pass | missing |
| [newtontech/mlip-lsp](https://github.com/newtontech/mlip-lsp) | `mlip` | `main` | `.mlip.json`, `.mlip.yaml`, `.mlip.yml`, `mlip.json`, `mlip.yaml`, `mlip.yml` | experimental | stable | `v0.2.0`<br>VERSION `0.2.0`<br>HEAD `74599859d2da` | pass | pass | pass | pass | pass | missing |
| [newtontech/pyatb-lsp](https://github.com/newtontech/pyatb-lsp) | `pyatb` | `main` | `.pyatb.py`, `run_pyatb.py` | experimental | stable | `v0.1.0`<br>VERSION `0.1.0`<br>HEAD `beacd3141616` | pass | pass | pass | pass | pass | missing |
| [newtontech/pyscf-lsp](https://github.com/newtontech/pyscf-lsp) | `pyscf` | `main` | `.pyscf.py`, `run_pyscf.py` | experimental | stable | `v0.1.0`<br>VERSION `0.1.0`<br>HEAD `766cf71b9e20` | pass | pass | pass | pass | pass | missing |
| [newtontech/dpgen-lsp](https://github.com/newtontech/dpgen-lsp) | `dpgen` | `main` | `param.json`, `machine.json` | experimental | stable | `v0.1.0` local-only<br>VERSION `0.1.0`<br>HEAD `89daa8c2d125` | pass | partial | pass | pass | pass | missing |

## Agent CLI Readiness

| Backend | Agent CLI | Operations | Help Smoke | Closed Loop |
|---------|-----------|------------|------------|-------------|
| `abacus-lsp` | `abacus-lsp-tool` | `check`, `context`, `complete`, `hover`, `symbols`, `fix` | pass | partial |
| `abinit-lsp` | `abinit-lsp-tool` | `check`, `context`, `complete`, `hover`, `symbols`, `fix` | pass | partial |
| `cif-lsp` | `cif-lsp-tool` | `check`, `context`, `complete`, `hover`, `symbols`, `fix` | pass | partial |
| `cp2k-lsp-enhanced` | `cp2k-lsp-tool` | `check`, `context`, `complete`, `hover`, `symbols`, `fix` | pass | partial |
| `vasp-lsp` | `vasp-lsp-tool` | `check`, `context`, `complete`, `hover`, `symbols`, `fix` | pass | partial |
| `gaussian-lsp` | `gaussian-lsp-tool` | `check`, `context`, `complete`, `hover`, `symbols`, `fix` | pass | partial |
| `orca-lsp` | `orca-lsp-tool` | `check`, `context`, `complete`, `hover`, `symbols`, `fix` | pass | partial |
| `qe-lsp` | `qe-lsp-tool` | `check`, `context`, `complete`, `hover`, `symbols`, `fix` | pass | partial |
| `gamess-lsp` | `gamess-lsp-tool` | `check`, `context`, `complete`, `hover`, `symbols`, `fix` | pass | partial |
| `nwchem-lsp` | `nwchem-lsp-tool` | `check`, `context`, `complete`, `hover`, `symbols`, `fix` | pass | partial |
| `gpumd-lsp` | `gpumd-lsp-tool` | `check`, `context`, `complete`, `hover`, `symbols`, `fix` | pass | partial |
| `gromacs-lsp` | `gromacs-lsp-tool` | `check`, `context`, `complete`, `hover`, `symbols`, `fix` | pass | partial |
| `lammps-lsp` | `lammps-lsp-tool` | `check`, `context`, `complete`, `hover`, `symbols`, `fix` | pass | partial |
| `mlip-lsp` | `mlip-lsp-tool` | `check`, `context`, `complete`, `hover`, `symbols`, `fix` | pass | partial |
| `pyatb-lsp` | `pyatb-lsp-tool` | `check`, `context`, `complete`, `hover`, `symbols`, `fix` | pass | partial |
| `pyscf-lsp` | `pyscf-lsp-tool` | `check`, `context`, `complete`, `hover`, `symbols`, `fix` | pass | partial |
| `dpgen-lsp` | `dpgen-lsp-tool` | `check`, `context`, `complete`, `hover`, `symbols`, `fix` | pass | partial |

## Actionable Gate Gaps

### abacus-lsp
- WARN: No docstring/wiki/raw traceability report found

### abinit-lsp
- WARN: No docstring/wiki/raw traceability report found

### cif-lsp
- WARN: No docstring/wiki/raw traceability report found

### cp2k-lsp-enhanced
- WARN: No docstring/wiki/raw traceability report found

### vasp-lsp
- WARN: No docstring/wiki/raw traceability report found

### gaussian-lsp
- WARN: No docstring/wiki/raw traceability report found

### orca-lsp
- WARN: No docstring/wiki/raw traceability report found

### qe-lsp
- WARN: No docstring/wiki/raw traceability report found

### gamess-lsp
- WARN: No docstring/wiki/raw traceability report found

### nwchem-lsp
- WARN: No docstring/wiki/raw traceability report found

### gpumd-lsp
- WARN: No docstring/wiki/raw traceability report found

### gromacs-lsp
- WARN: No docstring/wiki/raw traceability report found

### lammps-lsp
- WARN: No docstring/wiki/raw traceability report found

### mlip-lsp
- WARN: No docstring/wiki/raw traceability report found

### pyatb-lsp
- WARN: No docstring/wiki/raw traceability report found

### pyscf-lsp
- WARN: No docstring/wiki/raw traceability report found

### dpgen-lsp
- WARN: Latest local tag v0.1.0 is not present on origin
- WARN: No docstring/wiki/raw traceability report found
- Action: Push tag v0.1.0 to origin or retag the verified remote commit.


## OpenQC Integration Guarantees

Expand All @@ -58,25 +121,14 @@ Every standalone LSP exposes the same agent CLI operation set: `check`, `context
| Grammar contribution | Every language has a TextMate grammar under `syntaxes/`. |
| Configuration | Every language exposes `enabled`, `path`, `command`, `args`, and `env` settings under `openqc.lsp.<languageId>.*`. |
| Startup | OpenQC starts the configured command over stdio and can prefer sibling local repositories when no user override is set. |
| Latest tracking | Every registry entry records the upstream default branch used for latest-version checks. |
| Agent CLI probe | `npm run lsp:check-latest -- --fail-on-drift` verifies remote HEAD, local worktree cleanliness, and source-backed `*-lsp-tool --help`. |
| Bohrium registry alignment | `npm run lsp:check-bohrium-registry` compares OpenQC bundled ids/agent CLIs with `bohrium_skills/.../lsp_backends.yaml` and fails on drift. OpenQC editor integration and Bohrium skill routing are reported separately. |

## Release Verification

Before publishing a VSIX or Marketplace release:

1. Run `npm run typecheck`.
2. Run `npm run test:unit -- --runTestsByPath tests/unit/lsp/registry.test.ts`.
3. Run `npm run lsp:check-latest -- --fail-on-drift` to compare sibling LSP checkouts with each configured remote default-branch HEAD and verify source-backed `*-lsp-tool --help`.
4. Run `npm run lsp:check-bohrium-registry` to compare OpenQC bundled registry ids with the Bohrium skill backend registry.
5. Run `OpenQC LSP: Generate Compatibility Report` from VS Code and save the report with the release notes.
6. For each LSP, smoke test one valid file and one intentionally invalid file on the latest configured executable or sibling repository checkout.
7. Record the exact LSP version, release tag, or commit SHA used for the release smoke test.
| Latest tracking | `npm run lsp:check-latest -- --fail-on-drift` verifies local checkout cleanliness, remote HEAD parity, and agent CLI help. |
| Family maturity | `npm run lsp:check-family -- --strict` verifies manifest, provenance, fixture, smoke, and Diagnostic Engine readiness. |
| Traceability aggregation | `npm run lsp:check-family -- --strict` consumes backend docstring/wiki/raw traceability reports when present and reports missing or broken evidence links. |
| Bohrium routing | `npm run lsp:check-bohrium-registry` verifies OpenQC and Bohrium backend ids and agent CLI names stay aligned. |

## Known Limits

- The matrix records OpenQC integration support, not a promise that every standalone LSP implements every optional LSP method.
- `cp2k-lsp-enhanced` uses `develop` as its latest upstream branch.
- `cif-lsp` and `lammps-lsp` use `master` as their latest upstream branch.
- Experimental entries are wired for startup and language support, but may have narrower standalone server capabilities than the stable chemistry LSPs.
- This gate proves OpenQC integration readiness and the shared agent-facing contract; it does not prove exhaustive grammar coverage for every historical software release.
- Complete version-aware parser/rule coverage and official-doc ingestion remain tracked in each backend repository issue queue.
- Runtime scientific correctness still requires backend-specific fixture suites and, where available, real executable/log smoke tests.
- OpenQC launches the configured executable or sibling checkout; it does not vendor or pin standalone backend binaries.
1 change: 1 addition & 0 deletions package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1565,6 +1565,7 @@
"lsp:check-latest": "node scripts/check-lsp-latest.mjs",
"lsp:check-family": "node scripts/check-lsp-family.mjs",
"lsp:check-bohrium-registry": "node scripts/check-bohrium-registry-alignment.mjs --strict",
"lsp:generate-compatibility-doc": "node scripts/generate-lsp-compatibility-doc.mjs",
"pretest": "npm run compile && npm run lint",
"lint": "eslint src --ext ts",
"typecheck": "tsc --noEmit",
Expand Down
Loading
Loading