Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
84 changes: 84 additions & 0 deletions docs/issues/130-131-plan.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
# Issues 130, 131, and 142 plan

## Task

Keep syntax-context work bounded across repeated distant viewport jumps, preserve injected Rust highlighting for fenced Markdown nested in block quotes, and stop conservative Rust overflow highlighting at truncated-line context barriers.

## Current behavior

The windowed Rust and Markdown highlighter keeps line-zero plus eight recent context checkpoints.

Adding a checkpoint after a backward jump deletes every later checkpoint before enforcing the cap.

Alternating between distant viewports can therefore rescan from line zero or another distant checkpoint on every jump.

The Markdown checkpoint scanner recognizes fences only when the marker follows up to three leading spaces.

A fence such as `> ```rust` is missed, and a deep viewport cannot synthesize the quoted fence opener needed for injected Rust highlighting.

The #124 conservative fallback comment-classifies a complete rebuilt window when Rust block-comment nesting exceeds 64 levels.

That fallback bypasses the truncated-line barriers used by context scanning and the normal parser path.

Visible Rust code after an oversized line therefore remains incorrectly styled as a comment in plain Rust and fenced Markdown.

## Implementation

1. Keep line-zero plus a deterministic capped set of recently useful window-boundary checkpoints.
2. Insert and refresh checkpoints without deleting valid checkpoints on the other side of a jump.
3. Select the nearest valid checkpoint at or before a target independent of retention order.
4. Invalidate only checkpoints after the first changed line while retaining the context at the start of that line.
5. Recognize normalized block-quote container prefixes on Markdown fence openers.
6. Retain the quote depth in Markdown fence context and apply it consistently to synthetic seeds, fenced contents, and closing fences.
7. Leave a quoted fence when a later line no longer has its required container, then scan that line as ordinary Markdown.
8. Split plain Rust overflow fallback at each oversized-line barrier and resume ordinary Rust parsing after the barrier.
9. Stop fenced Markdown Rust overflow fallback at the first fence closer, oversized-line barrier, or loss of the required block-quote container.
10. Preserve the fixed Rust block-comment overflow state and its 64-level exact wrapper cap from #124.

## Acceptance criteria

- After two distant viewports are warmed, deterministic alternation between them scans and parses work bounded independently of file depth.
- Parser request bytes, retained highlight lines, and retained checkpoints keep deterministic caps.
- A backward jump does not discard still-valid forward checkpoints.
- An edit retains checkpoints at or before its first changed line and invalidates every later checkpoint whose syntax context may be stale.
- After the affected side is rebuilt once, alternating across an edit remains bounded.
- Shallow and deep block-quoted backtick and tilde fences retain Markdown fence styling and injected Rust highlighting.
- Quoted fence openers, contents, and closers use the same block-quote container depth.
- A quoted fence can close inside the visible window.
- Editing a quote container invalidates and rebuilds the affected context correctly.
- Plain Rust overflow fallback stops at the first truncated-line context barrier.
- Ordinary Rust parsing and highlighting resume after that barrier within the same rebuilt window.
- Fenced Markdown Rust overflow fallback stops at a truncated-line barrier without losing the surrounding quoted or unquoted fence context.
- Quoted Markdown overflow fallback ends when the required block-quote container disappears, including when that line is oversized.
- Edits before and around a truncated-line barrier invalidate and rebuild the affected window correctly.
- Ordinary unquoted fences and Rust comments, strings, raw strings, and character literals remain correct.
- The #124 nested Rust comment request bound and conservative overflow behavior remain unchanged in Rust and fenced Markdown.
- Parser input, scanned work, retained checkpoints, and cached window size remain within the #112 and #124 bounds after overflow recovery.

## Checks

- Add focused regressions for alternating distant Rust and Markdown viewports, including an edit between the locations.
- Assert scanned lines, request bytes, checkpoint count, and retained window size remain bounded after warm-up.
- Add shallow and deep quoted-fence regressions for backticks and tildes with injected Rust.
- Cover a quoted closer inside the visible window and edits that remove and restore a block-quote container.
- Add plain Rust and fenced Markdown overflow regressions with 65 openers, an oversized barrier line, visible Rust code after it, and edits before and around the barrier.
- Combine quoted overflow, a truncated-line barrier, and removal and restoration of the barrier's quote container.
- Run focused highlighter tests.
- Run every opt-in performance test with `cargo test performance:: -- --ignored --nocapture --test-threads=1`.
- Run `cargo fmt --check`.
- Run `git diff --check origin/main...HEAD`.
- Run `cargo test --all-targets`.
- Run `cargo clippy --all-targets --all-features -- -D warnings`.
- Run `cargo build --all-targets`.
- Apply the `improve` workflow to the final changed code without changing behavior.
- Obtain fresh independent subagent review and address every important finding.
- Wait for protected GitHub checks and current review feedback on the ready pull request.
- Reply with commit and pull-request evidence on issues #130, #131, and #142.
- Reply on the unresolved #141 review thread and resolve it only after the published fix fully addresses the report.

## Out of scope

- Background parsing.
- A new parser framework or complete CommonMark parser.
- LSP or semantic highlighting.
- Changes to rendering, themes, editor commands, or the #124 overflow policy.
Loading