Repository navigation
ppx_devtools.expect: handle repeated and unreached [%expect]s like ppx_expect - #6
Merged
Merged
Conversation
This was referenced Nov 3, 2026
hhugo
force-pushed
the
expect-uncaught-exn
branch
from
November 3, 2026 08:16
cc888ea to
4b9b01b
Compare
hhugo
force-pushed
the
expect-unreachable
branch
from
November 3, 2026 08:16
516f86f to
d2ffc3b
Compare
hhugo
force-pushed
the
expect-uncaught-exn
branch
from
November 3, 2026 08:42
4b9b01b to
cf19038
Compare
hhugo
force-pushed
the
expect-unreachable
branch
2 times, most recently
from
November 3, 2026 08:53
a65423d to
e183abe
Compare
hhugo
force-pushed
the
expect-uncaught-exn
branch
from
November 3, 2026 08:53
cf19038 to
713323f
Compare
…x_expect - An assertion point that runs several times with different outputs is no longer a runner error. Promotion records every output under a "CR expect_test: Test ran multiple times with different test outputs" comment, and the test keeps failing until it gives a single output. - A plain [%expect] that is never reached is promoted to [%expect.unreachable], which is now supported: when reached, it is promoted back to an [%expect] holding the output. A group with [@when] variants is left alone, as it may only be unreachable under some axes. Outputs are now recorded as the test runs and checked when it ends. With no runner errors left, a test can no longer be counted twice in the failure count. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An unreached group with [@when] variants used to be left alone. Now its active variant becomes an [%expect.unreachable] that keeps the condition, ([%expect.unreachable] [@when COND]), and the other variants are left for the axes under which the code may run. When reached, it is promoted back to [%expect ({| ... |} [@when COND])]. [%expect.unreachable] accordingly takes a [@when] attribute and is part of the group of the assertion points next to it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ect]s - An [%expect] that ran several times listed one output per run under its CR, repeats included. As in ppx_expect, it now lists each distinct output once, sorted. Only the distinct outputs are kept while the test runs. - An unreached group marked its default [%expect] unreachable even when a [@when] variant applied, e.g. ([%expect.unreachable] [@when native]). Code reached on some backends only then never converged, each backend's promotion undoing the other's. The default is now only marked when it is the active variant. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
hhugo
force-pushed
the
expect-unreachable
branch
from
November 3, 2026 09:03
e183abe to
6f606c1
Compare
- An [%expect] that saw different outputs passed under dune once its CR was promoted, as the corrected file then matched the source. Like ppx_expect, the runner now fails until the outputs agree, even under dune. - A reached [%expect.unreachable] with no output is reported as reached, rather than as an empty diff. - [mismatch] is defined with [report], and a [@when] is applied to a variant by a single [with_condition] helper. - [Corrected.Delete] is back, rather than [Replace ""]. - The CR lists each distinct output once, unlike ppx_expect, which lists every run: say so. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An [%expect] that saw different outputs made the runner fail under dune. Dune only diffs the [.corrected] files when the runner succeeds, so the CR, and every other correction of the run, could not be promoted. The runner now fails on such an [%expect] only once there is nothing left to promote, i.e. when no [.corrected] file differs from its source. Until then, the diffs fail the test. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The error for an [%expect] that saw different outputs gave the line of the first variant of the group, rather than of the variant applying in this environment, which is the one getting the CR. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Adjacent assertion points form a group of variants, of which the plain [%expect] is the default. A second plain [%expect] was put in the same group, where it was never selected: it was not checked, not marked unreachable when unreached, and a second [%expect.unreachable] was not converted when reached. A group now has a single default, and a second one starts the next group. As in ppx_expect, the second of two adjacent [%expect]s is thus checked against an empty output. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Run without [-diff-cmd -], the runner did not say that an [%expect] saw different outputs: once the CR was in the source, it only printed a diff with no change but the indentation. The error is now printed in both modes, and no correction is made for a CR that is already in the source, which removes that diff. Under dune, no [.corrected] file is written in that case, and the runner still fails. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…he CR - A second plain [%expect] started a new group, but a [@when] variant following a plain [%expect] still joined its group, rather than the group of the default after it. The default now ends its group: the assertion points following it form the next group. - The error for different outputs names what gets the CR: the variant applying in this environment, which may be an [%expect.unreachable], or the [%expect] to be inserted after the group. - The variant of a group is selected once, rather than twice. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
[check_groups] both dispatched on the outputs of a group and handled the unreached and different outputs cases, selecting a variant that [check_group] then matched on again. It now only dispatches, to [check_unreached] or to [check_reached] (the former [check_group]), which records the error for different outputs where it places the CR. No change in behaviour. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
In a group, the first [@when] variant whose condition holds applies, and otherwise the default, which ends the group. [@when] variants written after the default were thus silently made a group of their own, checked against an empty output. Such variants, with no default of their own, are now an error. The README states how a variant is selected. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
- A test whose body is a lone expression with an attribute, such as [print_string "x" [@attr]], did not preprocess ("Attributes not allowed here"), and the [%expect] added for its trailing output would have been inserted before the attribute, taking it over. The end of the body now covers its attributes. - [when_of_attributes] is defined with [when_condition], rather than duplicating it. - In [check_reached], the flag for different outputs no longer has the name of the function recording them. - The runs with and without [-diff-cmd -] of the test for a promoted CR share their expected output, which is the same. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
- [[%expect {| ... |}] [@when COND]] was silently an unconditional
[%expect], while the same form is a conditional variant with
[%expect.unreachable]. It now is one too, like
[[%expect ({| ... |} [@when COND])]].
- [Corrected.write] no longer tells whether a corrected file differs
from its source: every correction changes its source, now that none is
made for a CR that is already there.
- The function recording a correction is named [record], rather than
[report] like the one printing in [exit].
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
- The literal of a bare [%expect] was inserted one character before the end of the node, taken to be its closing bracket. In [([%expect] [@when COND])], this is the closing parenthesis, and promotion resulted in a syntax error. The literal is now inserted after the name of the extension. - [classify] handles a [@when] following the node in one place for [%expect] and [%expect.unreachable]. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
- Promoting [{%expect| ... |}] corrupted the source, as the literal of
this form has no delimiters of its own. The node is now rewritten as
an [[%expect {| ... |}]].
- A reached [%expect.unreachable] was rewritten together with the
[@when] following it, dropping any attribute or comment in between.
Only the name of the node is rewritten now, which keeps what
surrounds it: [[%expect.unreachable] [@when COND]] becomes
[[%expect {| ... |}] [@when COND]].
- Groups were numbered in the evaluation order of [::], so the errors
of a test came out in reverse source order.
- Run without [-diff-cmd -], a reached [%expect.unreachable] was only
said to be reached when it saw no output.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #5.
[%expect]that never runs (e.g. after an exception) is promoted to[%expect.unreachable], which fails when reached and is then promoted back to an[%expect]holding the output. In a group with[@when]variants, only the default becomes unreachable; the variants keep their snapshots, as the code may run under other axes.[%expect.unreachable]can itself be a[@when]variant.With no runner errors left in
-diff-cmd -mode, every failure goes through dune's diff and promotion, and a test is counted once in the standalone failure count.test/promote/disagree.ml, which checked for the runner error, is folded intopromote_me.ml.🤖 Generated with Claude Code