Goto def and find ref doesn't work for some modules in constantine

[moigagoo]

Reported on Reddit: https://www.reddit.com/r/nim/comments/1v2o02f/comment/oyycfg0/

[moigagoo]

OK, so here are the actual problems that need to be addressed to fix nimlangserver for Constantine and other projects with similar properties. I'll use the first lookup issue reported by the user (@advaita-saha):

https://github.com/mratsim/constantine/blob/master/constantine/math/elliptic/ec_scalar_mul.nim#L265

1. This call is inside a generic proc, so the exact overload can only be determined when the containing proc is instantiated with concrete types. There may be multiple valid instantiations that resolve the call to different overloads. Nimsuggest should return all definitions recorded for that source position instead of arbitrarily selecting one. LSP supports multiple definition locations, and nimlangserver can pass them to the editor, which can show a picker similar to the one used for find references.

2. Adding constraints to the generic parameters can make the call resolvable in some cases, for example changing EC to EC: EC_ShortW_Prj and ECaff to ECaff: EC_ShortW_Aff. However, partially constrained generics can still leave an unresolved overload set. Nimsuggest used to handle such a set by returning its first candidate, which could be the wrong overload. The correct behaviour is to return a genuinely resolved candidate, all candidates recorded from concrete instantiations, or no result if the call cannot be resolved precisely.

3. Even when the constrained version works in standalone nimsuggest, nimlangserver may not find the definition because it uses protocol version 4, while the default command-line invocation of nimsuggest uses version 2. The v4 code path (executeNoHooksV3) does not invoke the speculative generic-body analysis used by the v2 path. It primarily queries the suggestion database populated during ordinary compilation and concrete generic instantiation. Therefore, if the selected project root does not instantiate the generic body, v4 has no definition to return.

4. Constantine is a multi-entry-point package rather than a package with one complete root entry point. nimble dump reports constantine.nim as its entry point, but that file does not exist. Consequently, nimlangserver falls back to using the opened file as the nimsuggest project root. That context includes the file and its imports, but not independent entry points that use it. With the default limit of one nimsuggest process, later files may also reuse the first project context.

   This matters for generic code whose overload resolution depends on concrete instantiations elsewhere. Nimsuggest already supports a projectMapping setting that associates source files with project roots. A mapping can target an existing suitable entry point, or a dedicated analysis root that imports representative public APIs. The target must actually instantiate the relevant generic code; mapping by itself cannot provide definitions for instantiations that it does not compile.

   Therefore, project-specific mappings are useful for Constantine and other multi-entry-point projects, but they do not replace the nimsuggest fixes. The fixes improve analysis of generic code that is not covered by the selected entry-point context, while the mappings provide the broader entry-point context needed for precise project-wide results.

5. Nimsuggest's speculative generic analysis used to semantic-analyse the stored generic AST in place. Semantic analysis rewrites the AST by resolving identifiers, calls, and types. When analysis failed because generic parameters were still unresolved, the failed nodes could be replaced with error nodes. This corrupted the stored AST.

   The corruption explains why a def query could work once, but a later use query would find no symbol. use itself did not cause the corruption; it walked the already-corrupted AST. Running outline reparsed and recompiled the module, restoring the original unresolved symbol choices, which made use work again.

   The fix is to perform speculative analysis on a copy of the generic body, in a temporary scope, and to preserve the suggestion database and diagnostics across the fallback recompile.