Skip to content

Add nf-core/eager fixture + compare-and-contrast scenario against iwc#1234#346

Open
jmchilton wants to merge 1 commit into
mainfrom
nextflow-eager-scenario
Open

Add nf-core/eager fixture + compare-and-contrast scenario against iwc#1234#346
jmchilton wants to merge 1 commit into
mainfrom
nextflow-eager-scenario

Conversation

@jmchilton

Copy link
Copy Markdown
Member

Adds nf-core/eager to the Nextflow fixture corpus and a NEXTFLOW → GALAXY pipeline scenario that holds the journey against a human-authored answer to the same conversion.

The fixture

Pinned to the DSL2 rewrite on dev (f77d2464), deliberately not to a release. eager's latest tag (2.5.3) is DSL1 (nextflow.enable.dsl=1, monolithic main.nf), which [[summarize-nextflow]] short-circuits by design — a tag pin would die at phase 1 and the scenario would test nothing. tag is optional in fixtures.yaml and several entries are already HEAD-pinned. Bump the SHA rather than adding a tag until 3.0 ships; the entry's notes say so.

Tier large: ~110 .nf files, ~45 nf-core modules, ~23 local subworkflows. Materializes and passes make fixtures-verify.

The scenario

galaxyproject/iwc#1234 is a human-authored eager-style aDNA workflow targeting the same source. It's a reference case, not a matching testeval.md keeps the gates, the comparison is advisory. The value is in where and why the two diverge.

Why it earns its keep: eager is a toolkit with ~15 independent run_*/skip_* branch groups, and expressing that optionality in Galaxy is the hard part of this conversion. The PR is a worked human example of paying that cost — 52 steps, 12 carrying a when, with 13 of 41 tool steps (~⅓) being pure param plumbing (7 map_param_value to build when booleans, 6 pick_value to rejoin branches). That ratio is the calibration this case exists to provide.

The expect: aims at that level of option coverage without prescribing the route: some meaningful part of the source's branch surface should reach the user as run-time choice, but which controls, how many, and by what mechanism are the journey's call. The reference's four controls are framed as calibration, not a quota — a different three, or the same coverage via optional inputs instead of when gating, passes; padding with toggles eager's schema doesn't motivate fails on intent-fidelity.

This also closes a loophole in the existing framing: "reach a stated scope decision" was satisfiable by declaring "scope: one linear path," passing the gate while producing a workflow no analyst would use.

Caveats recorded in the case

  • Version boundary. The PR's README cites nf-co.re/eager/2.5.3/ — the human worked from eager 2.x while the fixture must pin 3.x. Tool vocabulary overlaps almost entirely, so the contrast holds, but it is not a controlled A/B and some per-step differences will be version artifacts.
  • Not in the corpus. The PR is unmerged, so paleogenomics/adna-analysis doesn't exist at the pinned IWC SHA; compare-against-iwc-exemplar can't reach it and will settle on another exemplar. Supply the PR by URL per the URL-not-mirror principle — the .ga is not mirrored here.
  • Human ≠ correct. The PR is open and unreviewed. It registers both Sex.DetERRmine branches (and both ANGSD paths) as workflow outputs, so any run yields necessarily-empty outputs; a journey that avoids that isn't wrong for differing.

Incidental finding

The case surfaces a real gap in the pipeline spine, noted but not fixed here: there is no find-test-data phase, so implement-galaxy-workflow-test takes test-data-refs with nothing upstream producing it — the validator already warns (phases[8]). aDNA reference data is heavy and non-obvious, so this is a likely place for that gap to bite.

Testing

npm run validate on the main base: 0 errors, 194 warnings — identical to the pre-change baseline. make fixtures-verify passes with the new pin. Scenario content is prose with no frontmatter; the fixture entry is YAML — no code touched.

🤖 Generated with Claude Code

Pin nf-core/eager to the DSL2 rewrite on `dev` (f77d2464), not to a
release: latest tag 2.5.3 is DSL1, which summarize-nextflow
short-circuits by design, so a tag pin would die at phase 1.

Add a NEXTFLOW → GALAXY scenario holding the journey against
galaxyproject/iwc#1234, a human-authored eager-style aDNA workflow.
Reference case, not a matching test: eval.md keeps the gates, the
comparison is advisory.

expect: aims at eager's optionality reaching the user as run-time
choice, without prescribing which controls or what mechanism. Closes
a loophole where "scope: one linear path" satisfied the gate while
producing an unusable workflow.

Caveats recorded: PR targets eager 2.x (version boundary, not a
controlled A/B); PR unmerged so absent from the pinned IWC SHA and
unreachable by compare-against-iwc-exemplar; PR unreviewed, so
divergence isn't automatically failure.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant