Skip to content

Reject ill-formed template bodies in BestOverloadFunctionMatch and capture their diagnostics - #1179

Open
bendavid wants to merge 2 commits into
compiler-research:mainfrom
bendavid:reject-ill-formed-overload-match
Open

bendavid wants to merge 2 commits into
compiler-research:mainfrom
bendavid:reject-ill-formed-overload-match

Conversation

@bendavid

@bendavid bendavid commented Nov 8, 2026 •

Copy link
Copy Markdown

Description

When the best match found by BestOverloadFunctionMatch is a function template specialization whose body hasn't been instantiated yet, this instantiates the body and rejects the match if the body is ill-formed. The diagnostics from that instantiation are captured instead of printed, and made available through a new API, GetLastOverloadMatchDiagnostics().

Motivation: in ROOT's Python bindings (cppjit, built on CppInterOp since ROOT 6.42), calling a function template with arguments whose instantiation fails currently either:

  • runs code that shouldn't have compiled (for example, a failing static_assert leaves the specialization valid), or
  • fails later when the call wrapper is compiled or linked, with the actual compiler error printed somewhere unrelated or not at all.

With this change, the caller gets "no viable function" plus the compiler errors to report. Warnings from a successful instantiation can be reported too. This is the CppInterOp part of a long-standing ROOT effort, see root-project/root#12449.

Details:

  • The match is rejected if any error occurred during the body instantiation (DiagnosticErrorTrap), not only if the specialization was marked invalid. Errors such as a failing static_assert or an error in a nested instantiation leave the specialization valid.
  • The rejected specialization is marked invalid, so later lookups reject it too, and InstantiateTemplate no longer returns a specialization marked invalid.
  • DefinitionRequired stays false: a pattern without a body (defined in a library) is still a valid match.
  • The diagnostics are rendered as clang prints them, including source snippets and "in instantiation of" notes. Printing still makes sense for a caller showing them to a user, but probing for a match stays quiet.

Relation to ROOT's vendored CppInterOp: ROOT carries the first part of this (rejecting ill-formed bodies, with diagnostics suppressed) as [cppinterop] Reject ill-formed instantiations in BestOverloadFunctionMatch [upstream] on the ROOT-cppjit-* branches. This PR is that patch rebased onto the reworked BestOverloadFunctionMatch from #1147, plus capturing the diagnostics instead of discarding them.

Open design question: there's now the Result<T> / ErrorRef error model with drained diagnostics (#1034). Returning the diagnostics through it (e.g. as an overload/deduction payload, once BestOverloadFunctionMatch migrates) would fit better than a separate thread-local string. I kept a minimal string API here since the structured diagnostics don't carry clang's rendered snippets and instantiation notes, which are what's most useful to show a user. I'm happy to rework it if you prefer a different approach.

Fixes # (none)

Type of change

  • Bug fix
  • New feature
  • Requires documentation updates

Testing

Added FunctionReflection_BestOverloadFunctionMatchIllFormedBody. It covers:

  • a failing static_assert: rejected, error reported, still rejected on a second lookup and by InstantiateTemplate;
  • an error in a nested class-template member instantiation: rejected, error reported;
  • a deprecation warning: the match is accepted and the warning reported;
  • an already-defined match and a clean match: no diagnostics.

Built against LLVM 23.1.6 (apt.llvm.org packages, Ubuntu 24.04) and ran check-cppinterop:

  • CppInterOpTests: 522 passed. The only 2 failures (Interpreter_DetectSystemCompilerIncludePaths, in- and out-of-process) also fail on unmodified main in the same environment.
  • DynamicLibraryManagerTests and TracingTests pass.

The equivalent change on ROOT's vendored copy, together with its use in ROOT's cppjit bindings, is in root-project/root#24053. There, all 65 cppjit/pyroot tests pass, including new Python tests for the reported diagnostics.

Checklist

  • I have read the contribution guide recently

🤖 Generated with Claude Code

…pture their diagnostics

When the best match is a not-yet-instantiated function template specialization, instantiate its
body and reject the match if the body is ill-formed. Callers use BestOverloadFunctionMatch to probe
whether a valid instantiation exists, so an ill-formed body should surface as "no viable function"
rather than as a hard error when a call wrapper is compiled for it later. This mirrors cling's
LookupHelper::overloadFunctionSelector.

The match is rejected if any error occurred during the instantiation (using a DiagnosticErrorTrap),
not only if the specialization itself was marked invalid: errors such as a failing static_assert or
an error in a nested instantiation leave the specialization valid, and accepting it would either run
code which should not have compiled, or fail at link time for the missing nested definition. The
rejected specialization is marked invalid, so that later lookups reject it as well, and
InstantiateTemplate no longer returns a specialization marked invalid.

The diagnostics from instantiating the body are rendered into a string as clang would print them
(with source snippets and instantiation notes) instead of being printed, and are available through
the new GetLastOverloadMatchDiagnostics, so that callers can report them where appropriate (e.g. the
errors when no valid match is found, or warnings from a successful instantiation), while probing for
a valid match stays quiet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@codecov

codecov Bot commented Nov 8, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 96.96970% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 88.88%. Comparing base (3a6e86f) to head (c1259b3).

Files with missing lines Patch % Lines
lib/CppInterOp/CppInterOp.cpp 96.96% 1 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1179      +/-   ##
==========================================
+ Coverage   88.86%   88.88%   +0.02%     
==========================================
  Files          24       24              
  Lines        7014     7047      +33     
==========================================
+ Hits         6233     6264      +31     
- Misses        781      783       +2     
Files with missing lines Coverage Δ
lib/CppInterOp/CppInterOp.cpp 91.25% <96.96%> (+0.02%) ⬆️
Files with missing lines Coverage Δ
lib/CppInterOp/CppInterOp.cpp 91.25% <96.96%> (+0.02%) ⬆️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

/// of passing them to the installed consumer. Like clangSilent this keeps
/// probing operations quiet, but the diagnostics remain available to report
/// where appropriate.
class clangCapture {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: class 'clangCapture' defines a non-default destructor but does not define a copy constructor, a copy assignment operator, a move constructor or a move assignment operator [cppcoreguidelines-special-member-functions]

ropriate.
                ^


// Diagnostics from the instantiation of the body of the best match in the last
// call to BestOverloadFunctionMatch on this thread
static thread_local std::string LastOverloadMatchDiagnostics;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: variable 'LastOverloadMatchDiagnostics' is non-const and globally accessible, consider making it const [cppcoreguidelines-avoid-non-const-global-variables]

atch on this thread
                                                    ^

)";
GetAllTopLevelDecls(code, Decls);

auto Candidates = [&Decls](llvm::StringRef name) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: no header providing "llvm::StringRef" is directly included [misc-include-cleaner]

unittests/CppInterOp/FunctionReflectionTest.cpp:14:

- #include <llvm/Support/Casting.h>
+ #include <llvm/ADT/StringRef.h>
+ #include <llvm/Support/Casting.h>

auto Candidates = [&Decls](llvm::StringRef name) {
std::vector<Cpp::FuncRef> candidates;
for (auto* D : Decls)
if (auto* FTD = llvm::dyn_cast<FunctionTemplateDecl>(D);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: no header providing "clang::FunctionTemplateDecl" is directly included [misc-include-cleaner]

unittests/CppInterOp/FunctionReflectionTest.cpp:12:

- #include <cstdint>
+ #include <clang/AST/DeclTemplate.h>
+ #include <cstdint>

…s, address clang-tidy

Enable -Wdeprecated-declarations explicitly around the declarations whose instantiation is expected
to warn, since cling disables it by default on Windows (where the test failed). Delete the copy and
move operations of clangCapture, mark the thread-local diagnostics string as an intended non-const
global, and include the headers used directly by the test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@bendavid

bendavid commented Nov 8, 2026

Copy link
Copy Markdown
Author

The three cppjit jobs (ubu24-x86-llvm22, ubu24-x86-llvm23-vg, osx26-arm-llvm23) fail on a single test, test_templates.py::TestTEMPLATES::test19_templated_operator_add. The failure is expected with this change, and needs a fix in the cppjit test itself.

The test's operator+ of CustomVec<V> reads the private member fX of a different specialization (CustomVec<int> from CustomVec<double>). Without a friend declaration that is ill-formed, and gcc and clang reject it when the body is instantiated. It only passed so far because BestOverloadFunctionMatch didn't reject matches whose instantiated body contains errors, which is what this PR changes.

The fix in compiler-research/cppjit is one line in the test:

         namespace OperatorAddTest {
         template <class V>
         class CustomVec {
+            template <class> friend class CustomVec;
             V fX;

It's valid independently of this PR, since it only grants the access the code already relies on. ROOT's vendored copy of the test has the same fix in root-project/root#24053. I haven't opened a cppjit PR yet.

The other failures don't come from this PR: the two WASM (Emscripten) jobs also fail on recent main runs, and ubu24-x86-gcc14-cling-root stopped during its ROOT build step without a log. The Windows cling failure of the new test was real (cling disables -Wdeprecated-declarations there), and is fixed in c1259b3.

@github-actions

github-actions Bot commented Nov 8, 2026

Copy link
Copy Markdown
Contributor

clang-tidy review says "All clean, LGTM! 👍"

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant