Caching & Cache invalidation

[kevinramharak]

There several layers of caching and cache invalidation which should be implemented correctly:

• Diagnostics
• Tree Sitter
  • AST's
  • Query's
• (custom) Text Documents

Diagnostics

Full report vs Unchanged

The LSP allows servers to send back Unchanged reports for diagnostics.

TextDocument changes

When a document is opened by the client, changes will be send with incremental version id's. The easiest way to implement the report kinds and caching is probably to version tag everything for open documents.

There should also be a code branch for when parsing fails after an edit on an open document. We should probably send an unchanged report to make sure diagnostics stay put when the user is typing and introduces unrecoverable parse errors.

Most diagnostics will happen with the pull model, but async long running checks (like redos detection) can be send later with the push/publish model. This should be thought through as push diagnostics are 'owned' by the server and should be cleared on changes.

TODO: don't send these as seperate push diagnostics for the file, but as part of workspace diagnostics

NOTE: it is probably best to keep the diagnostics of non-open files as part of the workspace diagnostics, and use the pull model only for open files. This would leave us to only 'own' the workspace diagnostics.

Refresh notification

The client might support workspace/diagnostic/refresh requests. This will trigger a refetch on all document and workspace diagnostics.

Keep this in mind for implementing diagnostics.

Tree Sitter

Refer to the [tree-sitter-web documentation](https://github.com/tree-sitter/tree-sitter/tree/master/lib/binding_wexs b).

TODO: there are a bunch of tree sitter API results that have a delete function, as they allocate memory in the WASM binary/runtime. These should be disposed properly.

AST

Tree Sitter has an 'edit' API which align well with VS Code's document changes model. Parse results for open documents should be cached and the changes for documents should be propagated to the matching tree. This should make reparsing very cheap which is preferable when the user is actively editing a document.

For non-open documents we should consider a (seperate) LRU cache, and maybe even figure out if these can be shared with open documents tree's. This would be beneficial if a workspace diagnostics is triggered on a non-open file, and the user opens that file triggering the client to 'open' the document.

Queries

A TS Query is marshalled to the wasm side and has to be cleaned up with delete(). We should consider keeping these object around to skip initialization and allocation in the wasm binary/runtime.

(custom) Text Documents

Regex Radar uses the TextDocuments manager from vscode-languageserver-node with TextDocument from vscode-languageserver-textdocument. This will manage the open documents for us, non-open documents are read from the file system and wrapped in a TextDocument. These documents (just as their parse results) should probably be cached with a small LRU cache.