Add nf-core/eager fixture + compare-and-contrast scenario against iwc#1234#346
Open
jmchilton wants to merge 1 commit into
Open
Add nf-core/eager fixture + compare-and-contrast scenario against iwc#1234#346jmchilton wants to merge 1 commit into
jmchilton wants to merge 1 commit into
Conversation
Pin nf-core/eager to the DSL2 rewrite on `dev` (f77d2464), not to a release: latest tag 2.5.3 is DSL1, which summarize-nextflow short-circuits by design, so a tag pin would die at phase 1. Add a NEXTFLOW → GALAXY scenario holding the journey against galaxyproject/iwc#1234, a human-authored eager-style aDNA workflow. Reference case, not a matching test: eval.md keeps the gates, the comparison is advisory. expect: aims at eager's optionality reaching the user as run-time choice, without prescribing which controls or what mechanism. Closes a loophole where "scope: one linear path" satisfied the gate while producing an unusable workflow. Caveats recorded: PR targets eager 2.x (version boundary, not a controlled A/B); PR unmerged so absent from the pinned IWC SHA and unreachable by compare-against-iwc-exemplar; PR unreviewed, so divergence isn't automatically failure. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
nf-core/eagerto the Nextflow fixture corpus and a NEXTFLOW → GALAXY pipeline scenario that holds the journey against a human-authored answer to the same conversion.The fixture
Pinned to the DSL2 rewrite on
dev(f77d2464), deliberately not to a release. eager's latest tag (2.5.3) is DSL1 (nextflow.enable.dsl=1, monolithicmain.nf), which [[summarize-nextflow]] short-circuits by design — a tag pin would die at phase 1 and the scenario would test nothing.tagis optional infixtures.yamland several entries are already HEAD-pinned. Bump the SHA rather than adding ataguntil 3.0 ships; the entry'snotessay so.Tier
large: ~110.nffiles, ~45 nf-core modules, ~23 local subworkflows. Materializes and passesmake fixtures-verify.The scenario
galaxyproject/iwc#1234 is a human-authored eager-style aDNA workflow targeting the same source. It's a reference case, not a matching test —
eval.mdkeeps the gates, the comparison is advisory. The value is in where and why the two diverge.Why it earns its keep: eager is a toolkit with ~15 independent
run_*/skip_*branch groups, and expressing that optionality in Galaxy is the hard part of this conversion. The PR is a worked human example of paying that cost — 52 steps, 12 carrying awhen, with 13 of 41 tool steps (~⅓) being pure param plumbing (7map_param_valueto buildwhenbooleans, 6pick_valueto rejoin branches). That ratio is the calibration this case exists to provide.The
expect:aims at that level of option coverage without prescribing the route: some meaningful part of the source's branch surface should reach the user as run-time choice, but which controls, how many, and by what mechanism are the journey's call. The reference's four controls are framed as calibration, not a quota — a different three, or the same coverage via optional inputs instead ofwhengating, passes; padding with toggles eager's schema doesn't motivate fails on intent-fidelity.This also closes a loophole in the existing framing: "reach a stated scope decision" was satisfiable by declaring "scope: one linear path," passing the gate while producing a workflow no analyst would use.
Caveats recorded in the case
nf-co.re/eager/2.5.3/— the human worked from eager 2.x while the fixture must pin 3.x. Tool vocabulary overlaps almost entirely, so the contrast holds, but it is not a controlled A/B and some per-step differences will be version artifacts.paleogenomics/adna-analysisdoesn't exist at the pinned IWC SHA;compare-against-iwc-exemplarcan't reach it and will settle on another exemplar. Supply the PR by URL per the URL-not-mirror principle — the.gais not mirrored here.Sex.DetERRminebranches (and both ANGSD paths) as workflow outputs, so any run yields necessarily-empty outputs; a journey that avoids that isn't wrong for differing.Incidental finding
The case surfaces a real gap in the pipeline spine, noted but not fixed here: there is no
find-test-dataphase, soimplement-galaxy-workflow-testtakestest-data-refswith nothing upstream producing it — the validator already warns (phases[8]). aDNA reference data is heavy and non-obvious, so this is a likely place for that gap to bite.Testing
npm run validateon themainbase: 0 errors, 194 warnings — identical to the pre-change baseline.make fixtures-verifypasses with the new pin. Scenario content is prose with no frontmatter; the fixture entry is YAML — no code touched.🤖 Generated with Claude Code