Skip to content

Commit 2d51076

Browse files
horenmarhorenmar
committed
Fix bad error reporting for nested exceptions in default configuration
Bad handling of `TestFailureException` when translating unexpected exceptions inside assertion macros led to the unexpected exceptions handling erroring out through throwing the same exception again. This was then backstopped by the machinery for handling uncaught exceptions from assertions, which is normally used by the `CATCH_CONFIG_FAST_COMPILE` machinery, where we assume that it can only be invoked because the assertion macros are not configured to catch assertions. Closes #1292
1 parent fee8162 commit 2d51076

20 files changed

Lines changed: 419 additions & 23 deletions

‎src/catch2/benchmark/catch_benchmark.hpp‎

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -86,9 +86,7 @@ namespace Catch {
8686
auto analysis = Detail::analyse(*cfg, samples.data(), samples.data() + samples.size());
8787
BenchmarkStats<> stats{ CATCH_MOVE(info), CATCH_MOVE(analysis.samples), analysis.mean, analysis.standard_deviation, analysis.outliers, analysis.outlier_variance };
8888
getResultCapture().benchmarkEnded(stats);
89-
} CATCH_CATCH_ANON (TestFailureException const&) {
90-
getResultCapture().benchmarkFailed("Benchmark failed due to failed assertion"_sr);
91-
} CATCH_CATCH_ALL{
89+
} CATCH_CATCH_ALL {
9290
getResultCapture().benchmarkFailed(translateActiveException());
9391
// We let the exception go further up so that the
9492
// test case is marked as failed.

‎src/catch2/internal/catch_exception_translator_registry.cpp‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -59,10 +59,10 @@ namespace Catch {
5959
// To avoid having to handle TFE explicitly everywhere, we just
6060
// rethrow it so that it goes back up the caller.
6161
catch( TestFailureException& ) {
62-
std::rethrow_exception(std::current_exception());
62+
return "{ nested assertion failed }";
6363
}
6464
catch( TestSkipException& ) {
65-
std::rethrow_exception(std::current_exception());
65+
return "{ nested SKIP() called }";
6666
}
6767
catch( std::exception const& ex ) {
6868
return ex.what();

‎tests/SelfTest/Baselines/automake.sw.approved.txt‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -95,6 +95,8 @@ Nor would this
9595
:test-result: PASS Approximate comparisons with mixed numeric types
9696
:test-result: PASS Arbitrary predicate matcher
9797
:test-result: PASS Assertion macros support bit operators and bool conversions
98+
:test-result: XFAIL Assertions can be nested - CHECK
99+
:test-result: XFAIL Assertions can be nested - REQUIRE
98100
:test-result: PASS Assertions then sections
99101
:test-result: PASS Basic use of the Contains range matcher
100102
:test-result: PASS Basic use of the Empty range matcher

‎tests/SelfTest/Baselines/automake.sw.multi.approved.txt‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -93,6 +93,8 @@
9393
:test-result: PASS Approximate comparisons with mixed numeric types
9494
:test-result: PASS Arbitrary predicate matcher
9595
:test-result: PASS Assertion macros support bit operators and bool conversions
96+
:test-result: XFAIL Assertions can be nested - CHECK
97+
:test-result: XFAIL Assertions can be nested - REQUIRE
9698
:test-result: PASS Assertions then sections
9799
:test-result: PASS Basic use of the Contains range matcher
98100
:test-result: PASS Basic use of the Empty range matcher

‎tests/SelfTest/Baselines/compact.sw.approved.txt‎

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -332,6 +332,11 @@ Compilation.tests.cpp:<line number>: passed: !(lhs & rhs) for: !(Val: 1 & Val: 2
332332
Compilation.tests.cpp:<line number>: passed: HasBitOperators{ 1 } & HasBitOperators{ 1 } for: Val: 1 & Val: 1
333333
Compilation.tests.cpp:<line number>: passed: lhs ^ rhs for: Val: 1 ^ Val: 2
334334
Compilation.tests.cpp:<line number>: passed: !(lhs ^ lhs) for: !(Val: 1 ^ Val: 1)
335+
AssertionHandler.tests.cpp:<line number>: failed: i > 10 for: 2 > 10
336+
AssertionHandler.tests.cpp:<line number>: failed: unexpected exception with message: '{ nested assertion failed }'; expression was: foo( 2 ) == 2
337+
AssertionHandler.tests.cpp:<line number>: passed: true
338+
AssertionHandler.tests.cpp:<line number>: failed: i > 10 for: 2 > 10
339+
AssertionHandler.tests.cpp:<line number>: failed: unexpected exception with message: '{ nested assertion failed }'; expression was: foo( 2 ) == 2
335340
Tricky.tests.cpp:<line number>: passed: true
336341
Tricky.tests.cpp:<line number>: passed: true
337342
Tricky.tests.cpp:<line number>: passed: true
@@ -2862,7 +2867,7 @@ InternalBenchmark.tests.cpp:<line number>: passed: med == 18. for: 18.0 == 18.0
28622867
InternalBenchmark.tests.cpp:<line number>: passed: q3 == 23. for: 23.0 == 23.0
28632868
Misc.tests.cpp:<line number>: passed:
28642869
Misc.tests.cpp:<line number>: passed:
2865-
test cases: 428 | 313 passed | 95 failed | 6 skipped | 14 failed as expected
2866-
assertions: 2281 | 2089 passed | 157 failed | 35 failed as expected
2870+
test cases: 430 | 313 passed | 95 failed | 6 skipped | 16 failed as expected
2871+
assertions: 2286 | 2090 passed | 157 failed | 39 failed as expected
28672872

28682873

‎tests/SelfTest/Baselines/compact.sw.multi.approved.txt‎

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -330,6 +330,11 @@ Compilation.tests.cpp:<line number>: passed: !(lhs & rhs) for: !(Val: 1 & Val: 2
330330
Compilation.tests.cpp:<line number>: passed: HasBitOperators{ 1 } & HasBitOperators{ 1 } for: Val: 1 & Val: 1
331331
Compilation.tests.cpp:<line number>: passed: lhs ^ rhs for: Val: 1 ^ Val: 2
332332
Compilation.tests.cpp:<line number>: passed: !(lhs ^ lhs) for: !(Val: 1 ^ Val: 1)
333+
AssertionHandler.tests.cpp:<line number>: failed: i > 10 for: 2 > 10
334+
AssertionHandler.tests.cpp:<line number>: failed: unexpected exception with message: '{ nested assertion failed }'; expression was: foo( 2 ) == 2
335+
AssertionHandler.tests.cpp:<line number>: passed: true
336+
AssertionHandler.tests.cpp:<line number>: failed: i > 10 for: 2 > 10
337+
AssertionHandler.tests.cpp:<line number>: failed: unexpected exception with message: '{ nested assertion failed }'; expression was: foo( 2 ) == 2
333338
Tricky.tests.cpp:<line number>: passed: true
334339
Tricky.tests.cpp:<line number>: passed: true
335340
Tricky.tests.cpp:<line number>: passed: true
@@ -2851,7 +2856,7 @@ InternalBenchmark.tests.cpp:<line number>: passed: med == 18. for: 18.0 == 18.0
28512856
InternalBenchmark.tests.cpp:<line number>: passed: q3 == 23. for: 23.0 == 23.0
28522857
Misc.tests.cpp:<line number>: passed:
28532858
Misc.tests.cpp:<line number>: passed:
2854-
test cases: 428 | 313 passed | 95 failed | 6 skipped | 14 failed as expected
2855-
assertions: 2281 | 2089 passed | 157 failed | 35 failed as expected
2859+
test cases: 430 | 313 passed | 95 failed | 6 skipped | 16 failed as expected
2860+
assertions: 2286 | 2090 passed | 157 failed | 39 failed as expected
28562861

28572862

‎tests/SelfTest/Baselines/console.std.approved.txt‎

Lines changed: 34 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -347,6 +347,38 @@ Exception.tests.cpp:<line number>: FAILED:
347347
due to unexpected exception with message:
348348
unexpected exception
349349

350+
-------------------------------------------------------------------------------
351+
Assertions can be nested - CHECK
352+
-------------------------------------------------------------------------------
353+
AssertionHandler.tests.cpp:<line number>
354+
...............................................................................
355+
356+
AssertionHandler.tests.cpp:<line number>: FAILED:
357+
REQUIRE( i > 10 )
358+
with expansion:
359+
2 > 10
360+
361+
AssertionHandler.tests.cpp:<line number>: FAILED:
362+
CHECK( foo( 2 ) == 2 )
363+
due to unexpected exception with message:
364+
{ nested assertion failed }
365+
366+
-------------------------------------------------------------------------------
367+
Assertions can be nested - REQUIRE
368+
-------------------------------------------------------------------------------
369+
AssertionHandler.tests.cpp:<line number>
370+
...............................................................................
371+
372+
AssertionHandler.tests.cpp:<line number>: FAILED:
373+
REQUIRE( i > 10 )
374+
with expansion:
375+
2 > 10
376+
377+
AssertionHandler.tests.cpp:<line number>: FAILED:
378+
REQUIRE( foo( 2 ) == 2 )
379+
due to unexpected exception with message:
380+
{ nested assertion failed }
381+
350382
-------------------------------------------------------------------------------
351383
Captures do not leave block with an exception
352384
-------------------------------------------------------------------------------
@@ -1672,6 +1704,6 @@ due to unexpected exception with message:
16721704
Why would you throw a std::string?
16731705

16741706
===============================================================================
1675-
test cases: 428 | 331 passed | 76 failed | 7 skipped | 14 failed as expected
1676-
assertions: 2260 | 2089 passed | 136 failed | 35 failed as expected
1707+
test cases: 430 | 331 passed | 76 failed | 7 skipped | 16 failed as expected
1708+
assertions: 2265 | 2090 passed | 136 failed | 39 failed as expected
16771709

‎tests/SelfTest/Baselines/console.sw.approved.txt‎

Lines changed: 37 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2566,6 +2566,41 @@ Compilation.tests.cpp:<line number>: PASSED:
25662566
with expansion:
25672567
!(Val: 1 ^ Val: 1)
25682568

2569+
-------------------------------------------------------------------------------
2570+
Assertions can be nested - CHECK
2571+
-------------------------------------------------------------------------------
2572+
AssertionHandler.tests.cpp:<line number>
2573+
...............................................................................
2574+
2575+
AssertionHandler.tests.cpp:<line number>: FAILED:
2576+
REQUIRE( i > 10 )
2577+
with expansion:
2578+
2 > 10
2579+
2580+
AssertionHandler.tests.cpp:<line number>: FAILED:
2581+
CHECK( foo( 2 ) == 2 )
2582+
due to unexpected exception with message:
2583+
{ nested assertion failed }
2584+
2585+
AssertionHandler.tests.cpp:<line number>: PASSED:
2586+
CHECK( true )
2587+
2588+
-------------------------------------------------------------------------------
2589+
Assertions can be nested - REQUIRE
2590+
-------------------------------------------------------------------------------
2591+
AssertionHandler.tests.cpp:<line number>
2592+
...............................................................................
2593+
2594+
AssertionHandler.tests.cpp:<line number>: FAILED:
2595+
REQUIRE( i > 10 )
2596+
with expansion:
2597+
2 > 10
2598+
2599+
AssertionHandler.tests.cpp:<line number>: FAILED:
2600+
REQUIRE( foo( 2 ) == 2 )
2601+
due to unexpected exception with message:
2602+
{ nested assertion failed }
2603+
25692604
-------------------------------------------------------------------------------
25702605
Assertions then sections
25712606
-------------------------------------------------------------------------------
@@ -19134,6 +19169,6 @@ Misc.tests.cpp:<line number>
1913419169
Misc.tests.cpp:<line number>: PASSED:
1913519170

1913619171
===============================================================================
19137-
test cases: 428 | 313 passed | 95 failed | 6 skipped | 14 failed as expected
19138-
assertions: 2281 | 2089 passed | 157 failed | 35 failed as expected
19172+
test cases: 430 | 313 passed | 95 failed | 6 skipped | 16 failed as expected
19173+
assertions: 2286 | 2090 passed | 157 failed | 39 failed as expected
1913919174

‎tests/SelfTest/Baselines/console.sw.multi.approved.txt‎

Lines changed: 37 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2564,6 +2564,41 @@ Compilation.tests.cpp:<line number>: PASSED:
25642564
with expansion:
25652565
!(Val: 1 ^ Val: 1)
25662566

2567+
-------------------------------------------------------------------------------
2568+
Assertions can be nested - CHECK
2569+
-------------------------------------------------------------------------------
2570+
AssertionHandler.tests.cpp:<line number>
2571+
...............................................................................
2572+
2573+
AssertionHandler.tests.cpp:<line number>: FAILED:
2574+
REQUIRE( i > 10 )
2575+
with expansion:
2576+
2 > 10
2577+
2578+
AssertionHandler.tests.cpp:<line number>: FAILED:
2579+
CHECK( foo( 2 ) == 2 )
2580+
due to unexpected exception with message:
2581+
{ nested assertion failed }
2582+
2583+
AssertionHandler.tests.cpp:<line number>: PASSED:
2584+
CHECK( true )
2585+
2586+
-------------------------------------------------------------------------------
2587+
Assertions can be nested - REQUIRE
2588+
-------------------------------------------------------------------------------
2589+
AssertionHandler.tests.cpp:<line number>
2590+
...............................................................................
2591+
2592+
AssertionHandler.tests.cpp:<line number>: FAILED:
2593+
REQUIRE( i > 10 )
2594+
with expansion:
2595+
2 > 10
2596+
2597+
AssertionHandler.tests.cpp:<line number>: FAILED:
2598+
REQUIRE( foo( 2 ) == 2 )
2599+
due to unexpected exception with message:
2600+
{ nested assertion failed }
2601+
25672602
-------------------------------------------------------------------------------
25682603
Assertions then sections
25692604
-------------------------------------------------------------------------------
@@ -19123,6 +19158,6 @@ Misc.tests.cpp:<line number>
1912319158
Misc.tests.cpp:<line number>: PASSED:
1912419159

1912519160
===============================================================================
19126-
test cases: 428 | 313 passed | 95 failed | 6 skipped | 14 failed as expected
19127-
assertions: 2281 | 2089 passed | 157 failed | 35 failed as expected
19161+
test cases: 430 | 313 passed | 95 failed | 6 skipped | 16 failed as expected
19162+
assertions: 2286 | 2090 passed | 157 failed | 39 failed as expected
1912819163

‎tests/SelfTest/Baselines/junit.sw.approved.txt‎

Lines changed: 33 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
<?xml version="1.0" encoding="UTF-8"?>
22
<testsuitesloose text artifact
33
>
4-
<testsuite name="<exe-name>" errors="17" failures="140" skipped="12" tests="2293" hostname="tbd" time="{duration}" timestamp="{iso8601-timestamp}">
4+
<testsuite name="<exe-name>" errors="17" failures="140" skipped="12" tests="2298" hostname="tbd" time="{duration}" timestamp="{iso8601-timestamp}">
55
<properties>
66
<property name="random-seed" value="1"/>
77
<property name="filters" value="&quot;*&quot; ~[!nonportable] ~[!benchmark] ~[approvals]"/>
@@ -390,6 +390,38 @@ at Exception.tests.cpp:<line number>
390390
<testcase classname="<exe-name>.global" name="Arbitrary predicate matcher/Function pointer" time="{duration}" status="run"/>
391391
<testcase classname="<exe-name>.global" name="Arbitrary predicate matcher/Lambdas + different type" time="{duration}" status="run"/>
392392
<testcase classname="<exe-name>.global" name="Assertion macros support bit operators and bool conversions" time="{duration}" status="run"/>
393+
<testcase classname="<exe-name>.global" name="Assertions can be nested - CHECK" time="{duration}" status="run">
394+
<skipped message="TEST_CASE tagged with !mayfail"/>
395+
<failure message="i > 10" type="REQUIRE">
396+
FAILED:
397+
REQUIRE( i > 10 )
398+
with expansion:
399+
2 > 10
400+
at AssertionHandler.tests.cpp:<line number>
401+
</failure>
402+
<error message="foo( 2 ) == 2" type="CHECK">
403+
FAILED:
404+
CHECK( foo( 2 ) == 2 )
405+
{ nested assertion failed }
406+
at AssertionHandler.tests.cpp:<line number>
407+
</error>
408+
</testcase>
409+
<testcase classname="<exe-name>.global" name="Assertions can be nested - REQUIRE" time="{duration}" status="run">
410+
<skipped message="TEST_CASE tagged with !mayfail"/>
411+
<failure message="i > 10" type="REQUIRE">
412+
FAILED:
413+
REQUIRE( i > 10 )
414+
with expansion:
415+
2 > 10
416+
at AssertionHandler.tests.cpp:<line number>
417+
</failure>
418+
<error message="foo( 2 ) == 2" type="REQUIRE">
419+
FAILED:
420+
REQUIRE( foo( 2 ) == 2 )
421+
{ nested assertion failed }
422+
at AssertionHandler.tests.cpp:<line number>
423+
</error>
424+
</testcase>
393425
<testcase classname="<exe-name>.global" name="Assertions then sections" time="{duration}" status="run"/>
394426
<testcase classname="<exe-name>.global" name="Assertions then sections/A section" time="{duration}" status="run"/>
395427
<testcase classname="<exe-name>.global" name="Assertions then sections/A section/Another section" time="{duration}" status="run"/>

0 commit comments

Comments
 (0)