perf(kzg): execute test vectors instead of proving each one#672
Open
shuklaayush wants to merge 5 commits into
Open
perf(kzg): execute test vectors instead of proving each one#672shuklaayush wants to merge 5 commits into
shuklaayush wants to merge 5 commits into
Conversation
test_multiple_valid_verify_kzg proved all 54 vectors end-to-end, rebuilding the guest and regenerating the app proving key per vector (~24 min in CI). The guest asserts the verification result, so pure execution carries the same pass/fail signal per vector; the proving pipeline stays covered by test_single_valid_verify_kzg. Build and compile the guest once per test, run each vector through sdk.execute, and assert invalid vectors fail via execution error. This makes test_multiple_invalid_verify_kzg cheap enough to un-ignore, so all 68 invalid vectors now run in CI. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Executing the vectors covers guest logic but not the circuits: trace generation and proving can fail on operand edge cases that execute fine. Add test_prove_multiple_valid_verify_kzg, which proves all 54 valid vectors with a shared prover (keygen once). Proving costs ~25s per vector on the 64-core runner, so the test is ignored by default and CI runs it on pushes to main and on PRs with the run-kzg-prove-all label. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Crate-agnostic name so the same label can gate ignored proving tests in other workflows. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
shuklaayush
added
run-kzg-proving
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adding the run-kzg-proving label invokes tests-kzg.yml via workflow_call, starting only the prove-all job. Other label additions no longer re-run the workflow, and the fast test job is skipped on label events. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
shuklaayush
added
run-kzg-proving
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The tests-kzg workflow takes 25–29 min per run, ~24 min of which is
test_multiple_valid_verify_kzg: it generated a full app proof for each of the 54 valid KZG vectors, recreating theSdk(guest build + app keygen) per vector on top of the ~25s/vector proving cost.Proving is the irreducible cost — measurement shows keygen+build is only a few seconds while each proof takes ~25s on the 64-core runner — so the fix is to not prove every vector on every PR, while keeping full proving coverage available.
What
test_multiple_valid_verify_kzgbuilds and compiles the guest once, then runs each vector through the SDK's pure-execution path (sdk.compile+sdk.execute) instead of proving. The guestassert!s the verification result, so execution carries the per-vector pass/fail signal for guest logic. 1438s → 4.1s.test_prove_multiple_valid_verify_kzg(#[ignore]d by default) proves all 54 valid vectors with a shared prover, so keygen runs once. This covers what execution alone cannot: circuit trace generation and proving over the full input range, which can fail on operand edge cases that execute fine. Verified passing locally over all 54 vectors.prove-alljob runs this test on pushes tomainand on PRs carrying therun-kzg-provinglabel. Following the openvm label-overrides pattern, a newlabel-overrides.ymlfires onlabeledevents and invokestests-kzg.ymlviaworkflow_call, so adding the label starts the proving job immediately (the fasttestjob is skipped on label events, and unrelated labels don't re-trigger anything).test_single_valid_verify_kzgstill proves one vector on every PR, keeping the proving pipeline smoke-tested.catch_unwind; this madetest_multiple_invalid_verify_kzg— previously#[ignore = "takes too long"]— cheap enough to un-ignore, so all 68 invalid vectors now run in CI (42s).Expected CI time: ~6 min on PRs (vs ~25 min), with full 54-vector proving on
mainpushes and label-opted PRs.🤖 Generated with Claude Code