AI Agent Guidelines for quantmsdiann Development

This document provides comprehensive guidance for AI agents working with the quantmsdiann bioinformatics pipeline. These guidelines ensure code quality, maintainability, and compliance with project standards.

Critical: Mandatory Validation Before ANY Commit

ALWAYS run pre-commit hooks before committing ANY changes:

pre-commit run --all-files

This is non-negotiable. All code must pass formatting and style checks before being committed.

Project Overview

quantmsdiann is a bigbio bioinformatics pipeline, built following nf-core guidelines, for DIA-NN-based quantitative mass spectrometry. It is a standalone pipeline focused exclusively on Data-Independent Acquisition (DIA) workflows using the DIA-NN search engine.

This pipeline does NOT support DDA, TMT, iTRAQ, LFQ-DDA, or any non-DIA workflows. Those are handled by the parent quantms pipeline.

Key Features:

Built with Nextflow DSL2
DIA-NN for peptide/protein identification and quantification
Supports DIA-NN v1.8.1, v2.1.0, v2.2.0, v2.3.2, and v2.5.0 (latest)
QuantUMS quantification method (DIA-NN >= 1.9.2)
Parquet-native output with decoy reporting (DIA-NN >= 2.0)
MSstats-compatible output generation (via quantms-utils conversion, no MSstats analysis)
Quality control with pmultiqc
Complies with nf-core standards

Repository: https://github.com/bigbio/quantmsdiann

Technology Stack

Core Technologies

Nextflow: >=25.10.4 (DSL2 syntax)
nf-schema plugin: 2.6.1 (parameter validation)
nf-test: Testing framework (config: nf-test.config)
nf-core tools: Pipeline standards and linting
Containers: Docker/Singularity/Apptainer/Podman (Conda deprecated)
DIA-NN: Primary search engine (versions 1.8.1 through 2.5.0)

Key Configuration Files

nextflow.config - Main pipeline configuration
nextflow_schema.json - Parameter schema (auto-generated)
nf-test.config - Testing configuration
.nf-core.yml - nf-core compliance settings
modules.json - Module dependencies
.pre-commit-config.yaml - Pre-commit hooks

Project Structure

quantmsdiann/
├── main.nf                    # Pipeline entry point
├── workflows/
│   ├── quantmsdiann.nf        # Main workflow orchestrator
│   └── dia.nf                 # DIA-NN analysis workflow
├── subworkflows/local/        # Reusable subworkflows
│   ├── input_check/           # SDRF validation
│   ├── file_preparation/      # Format conversion
│   └── create_input_channel/  # SDRF metadata parsing
├── modules/local/
│   ├── diann/                 # DIA-NN modules (7 steps)
│   │   ├── generate_cfg/
│   │   ├── insilico_library_generation/
│   │   ├── preliminary_analysis/
│   │   ├── assemble_empirical_library/
│   │   ├── individual_analysis/
│   │   ├── final_quantification/
│   │   └── diann_msstats/
│   ├── openms/                # mzML indexing, peak picking
│   ├── pmultiqc/              # QC reporting
│   ├── sdrf_parsing/          # SDRF parsing
│   ├── samplesheet_check/     # Input validation
│   └── utils/                 # Format converters & helpers
│       ├── decompress_dotd/   # Decompress Bruker .d archives
│       ├── mzml_statistics/   # MS1/MS2 statistics for mzML
│       └── wiff_convert/      # SCIEX .wiff -> mzML via WiffConverter
├── modules/bigbio/            # Shared bigbio modules (synced via nf-core)
│   ├── pridepy/               # PRIDE Archive raw-file downloader
│   ├── qpx/                   # QPX Parquet + MuData export
│   └── thermorawfileparser/   # Thermo .raw -> mzML
├── conf/
│   ├── base.config            # Resource definitions
│   ├── modules/               # Module-specific configs
│   ├── tests/                 # Test profile configs (DIA only)
│   └── diann_versions/        # DIA-NN version-override configs for merge matrix
├── tests/                     # nf-test test cases
└── assets/                    # Pipeline assets and schemas

DIA-NN Workflow

The pipeline executes the following steps:

(Optional) PRIDE Archive Download - Pre-fetches raw files from PRIDE via pridepy when --pridepy_download true is set (uses --project_accession and --pridepy_protocol)
SDRF Validation & Parsing - Validates input SDRF and extracts metadata
File Preparation - Converts Thermo .raw (ThermoRawFileParser) and SCIEX .wiff + .wiff.scan (WiffConverter) to mzML; .d (Bruker) and .dia files are passed through natively
Generate Config - Creates DIA-NN config from enzyme/modifications (quantmsutilsc dianncfg)
In-Silico Library Generation - Predicts spectral library from FASTA (or uses provided library)
Preliminary Analysis - Per-file calibration and mass accuracy determination
Assemble Empirical Library - Builds consensus library from preliminary results using .quant files
Individual Analysis - Per-file search with empirical library (optional, for large datasets)
Final Quantification - Summary quantification with protein/peptide/gene matrices
MSstats Format Conversion - Converts DIA-NN report to MSstats-compatible CSV (quantmsutilsc diann2msstats)
(Optional) QPX Export - Exports DIA-NN output to QPX Parquet dataset and MuData .h5mu when --enable_qpx_export is set
pmultiqc - Quality control reporting

DIA-NN Version-Specific Features

Feature	Min Version	Parameter
Core workflow, library-free, .quant caching	1.8.1	(default)
QuantUMS quantification	1.9.2	`--quantums true`
Parquet output format	2.0	(automatic in 2.0+)
Decoy reporting	2.0	`--report_decoys true`
Native .raw on Linux	2.1.0	(automatic)
DDA support	2.3.2	`--dda true`
InfinDIA	2.3.0	`--enable_infin_dia`
DL model fine-tuning & selection	2.5.0	`--extra_args`

Validation Workflow

1. Pre-commit Hooks (MANDATORY)

Installation:

pip install pre-commit
pre-commit install  # Install git hooks (one-time setup)

Run before EVERY commit:

pre-commit run --all-files

Configured Hooks (.pre-commit-config.yaml):

Prettier - Formats code consistently across multiple file types
trailing-whitespace - Removes trailing whitespace (preserves markdown linebreaks)
end-of-file-fixer - Ensures files end with a single newline

Auto-fix in CI: If you forget to run pre-commit locally, comment on your PR:

@nf-core-bot fix linting

2. Pipeline Linting (RECOMMENDED)

nf-core pipelines lint
# For master branch PRs:
nf-core pipelines lint --release

3. Schema Validation (REQUIRED for parameter changes)

nf-core pipelines schema build

Testing Strategy

3-Tier CI/CD Strategy

Every PR / push to dev: Test all features against latest DIA-NN (2.2.0) + test 1.8.1 for features it supports.
Merge dev → master: Run the full version × feature matrix — every DIA-NN version against every feature it introduced.
ci.yml (fast gate): Only 1.8.1 public container tests, no auth needed.

Test Profiles (DIA only)

Profile	Feature Tested	Default Container	Min DIA-NN
`test_dia`	Core workflow	biocontainers 1.8.1 (public)	1.8.1
`test_dia_dotd`	Bruker .d format	biocontainers 1.8.1 (public)	1.8.1
`test_dia_quantums`	QuantUMS quantification	ghcr.io/bigbio/diann:2.2.0	1.9.2
`test_dia_parquet`	Parquet output + decoys	ghcr.io/bigbio/diann:2.2.0	2.0
`test_dia_qpx`	QPX Parquet + MuData export	biocontainers 1.8.1 (public)	1.8.1
`test_dia_skip_preanalysis`	`skip_preliminary_analysis` path	biocontainers 1.8.1 (public)	1.8.1
`test_dda`	DDA mode (`--dda true`, requires >= 2.3.2)	ghcr.io/bigbio/diann:2.3.2	2.3.2
`test_latest_dia`	Core on latest DIA-NN	ghcr.io/bigbio/diann:2.2.0	latest
`test_dia_2_2_0`	DIA-NN 2.2.0 compat	ghcr.io/bigbio/diann:2.2.0	2.2.0
`test_full_dia`	Full-size dataset	biocontainers 1.8.1 (public)	1.8.1

test_dia_local is not a standalone profile but a dev-container override (sdrf-pipelines / quantms-utils dev images). Stack it on a real test profile, e.g. -profile test_dia,test_dia_local,docker.

Version Override Profiles (for merge matrix)

These apply on top of test profiles to override the DIA-NN container version:

Profile	Container	Auth
`diann_v1_8_1`	`biocontainers/diann:v1.8.1_cv1`	none
`diann_v2_1_0`	`ghcr.io/bigbio/diann:2.1.0`	GHCR
`diann_v2_2_0`	`ghcr.io/bigbio/diann:2.2.0`	GHCR
`diann_v2_3_2`	`ghcr.io/bigbio/diann:2.3.2`	GHCR
`diann_v2_5_0`	`ghcr.io/bigbio/diann:2.5.0`	GHCR

CI Workflows

Workflow	Trigger	What it runs
ci.yml	Every PR (fast gate)	`test_dia`, `test_dia_dotd` (1.8.1, Docker)
extended_ci.yml	Every PR / push to dev	1.8.1 defaults + all features on 2.2.0 + Singularity
merge_ci.yml	PR to master / releases	Full version × feature matrix (10 combinations)
linting.yml	All PRs, releases	Pre-commit hooks + `nf-core pipelines lint`
branch.yml	PRs to master	Only allows PRs from `dev` branch

When to Run Tests Locally

No testing required:

README, CHANGELOG, docs/ updates
Minor config tweaks (labels, descriptions)
Comment additions

Targeted testing required:

Change Area	Test Profile	Command
Core DIA-NN modules	`test_dia`	`nextflow run . -profile test_dia,docker --outdir results`
Bruker .d support	`test_dia_dotd`	`nextflow run . -profile test_dia_dotd,docker --outdir results`
QuantUMS / final_quantification	`test_dia_quantums`	`nextflow run . -profile test_dia_quantums,docker --outdir results`
Parquet output / diann_msstats	`test_dia_parquet`	`nextflow run . -profile test_dia_parquet,docker --outdir results`
Cross-version compat	Use version override	`nextflow run . -profile test_dia,diann_v2_2_0,docker --outdir results`

Comprehensive testing (before PR):

nf-test test --profile debug,test,docker --verbose

Container Authentication

Tests using ghcr.io/bigbio/diann:* containers require GHCR authentication (DIA-NN has an academic-only license):

echo "$GHCR_TOKEN" | docker login ghcr.io -u $GHCR_USERNAME --password-stdin

In CI, the GHCR_USERNAME and GHCR_TOKEN secrets are configured in the repository.

Development Conventions

Branch Strategy

Target branch: dev (NOT master)
Master branch: Release-ready code only
PR process: Fork -> feature branch -> PR to dev

Naming Conventions

Channel Names

ch_output_from_<process_name>     // Initial output from a process
ch_<previous_process>_for_<next_process>  // Intermediate channels

Process/Module Names

Use lowercase with underscores: final_quantification, preliminary_analysis
Follow nf-core conventions for consistency

Resource Labels

Defined in conf/base.config:

Label	CPU	Memory	Time	Use Case
`process_single`	1	6 GB	4h	Single-threaded tools
`process_tiny`	1	1 GB	1h	Minimal processing
`process_very_low`	2	4 GB	3h	Light parallelism
`process_low`	4	12 GB	6h	Moderate workload
`process_medium`	8	36 GB	8h	Standard processing
`process_high`	12	72 GB	16h	Heavy computation

DIA-NN Module Labels

All DIA-NN process modules use the diann label for container selection:

process DIANN_FINAL_QUANTIFICATION {
    label 'process_high'
    label 'diann'
    // ...
}

The diann label is what version-override profiles target to switch containers.

Adding a New DIA-NN Feature

Identify minimum DIA-NN version that supports the feature
Modify the relevant module in modules/local/diann/
Add parameter to nextflow.config with sensible default
Update schema: nf-core pipelines schema build
Create or update test profile in conf/tests/ with the feature enabled
Add to CI matrix in extended_ci.yml (latest) and merge_ci.yml (version matrix)
Update documentation: docs/usage.md, docs/output.md

Code Style

Indentation: 4 spaces (enforced by Prettier)
Line length: Aim for <120 characters
Comments: Use // for single-line, /* */ for multi-line
Strings: Use single quotes 'text' unless interpolation needed "$var"
Groovy closures: Follow Nextflow DSL2 patterns

Troubleshooting

Pre-commit Issues

Problem: Pre-commit hook fails with formatting issues

Solution: The files were auto-fixed. Stage and commit again:

git add .
git commit -m "your message"

Testing Issues

Problem: Test fails with "Process exceeded memory limit"

Solution: Ensure you're using a test profile with resource limits:

nextflow run . -profile test_dia,docker --outdir results

Problem: GHCR container pull fails

Solution: Feature test profiles require GHCR authentication:

echo "$GHCR_TOKEN" | docker login ghcr.io -u $GHCR_USERNAME --password-stdin

For local testing without GHCR access, use test_dia or test_dia_dotd (public containers).

Problem: Snapshot test fails after intentional output changes

Solution: Update snapshots:

nf-test test --profile debug,test,docker --update-snapshot

Nextflow Issues

Problem: "Nextflow version is too old"

Solution:

nextflow self-update
# Or install specific version
export NXF_VER=25.10.4

Problem: "Process terminated with exit code 137"

Solution: Out of memory. Either:

Use test profile: -profile test_dia,docker
Increase Docker memory limit in Docker Desktop settings

Problem: Process error

Solution:

Check .nextflow.log for details
Check work directory: cat work/<hash>/.command.err
Rerun with debug: nextflow run . -profile debug,test_dia,docker --outdir results

Quick Reference

Essential Commands

# Pre-commit (MANDATORY before commit)
pre-commit run --all-files

# Lint pipeline
nf-core pipelines lint

# Update schema after parameter changes
nf-core pipelines schema build

# Run core DIA test (public container, no auth)
nextflow run . -profile test_dia,docker --outdir results

# Run QuantUMS test (requires GHCR auth)
nextflow run . -profile test_dia_quantums,docker --outdir results

# Run with specific DIA-NN version override
nextflow run . -profile test_dia,diann_v2_2_0,docker --outdir results

# Run nf-test suite
nf-test test --profile debug,test,docker --verbose

# Resume pipeline
nextflow run . -profile test_dia,docker --outdir results -resume

# Clean work directory
nextflow clean -f

File Locations

Main config: nextflow.config
Schema: nextflow_schema.json
Pre-commit config: .pre-commit-config.yaml
nf-test config: nf-test.config
Test configs: conf/tests/*.config
Version overrides: conf/diann_versions/*.config
Module configs: conf/modules/modules.config
Base resources: conf/base.config
Main workflow: workflows/quantmsdiann.nf
DIA workflow: workflows/dia.nf
DIA-NN modules: modules/local/diann/*/main.nf
Entry point: main.nf

Last Updated: 2026-08-14 Pipeline Version: 2.3.0 Minimum Nextflow: 25.10.4

Navigation Menu

Uh oh!

AI Agent Guidelines for quantmsdiann Development

Critical: Mandatory Validation Before ANY Commit

Project Overview

Technology Stack

Core Technologies

Key Configuration Files

Project Structure

DIA-NN Workflow

DIA-NN Version-Specific Features

Validation Workflow

1. Pre-commit Hooks (MANDATORY)

2. Pipeline Linting (RECOMMENDED)

3. Schema Validation (REQUIRED for parameter changes)

Testing Strategy

3-Tier CI/CD Strategy

Test Profiles (DIA only)

Version Override Profiles (for merge matrix)

CI Workflows

When to Run Tests Locally

Container Authentication

Development Conventions

Branch Strategy

Naming Conventions

Channel Names

Process/Module Names

Resource Labels

DIA-NN Module Labels

Adding a New DIA-NN Feature

Code Style

Troubleshooting

Pre-commit Issues

Testing Issues

Nextflow Issues

Quick Reference

Essential Commands

File Locations

Uh oh!

FilesExpand file tree

AGENTS.md

Latest commit

History

AGENTS.md

File metadata and controls

AI Agent Guidelines for quantmsdiann Development

Critical: Mandatory Validation Before ANY Commit

Project Overview

Technology Stack

Core Technologies

Key Configuration Files

Project Structure

DIA-NN Workflow

DIA-NN Version-Specific Features

Validation Workflow

1. Pre-commit Hooks (MANDATORY)

2. Pipeline Linting (RECOMMENDED)

3. Schema Validation (REQUIRED for parameter changes)

Testing Strategy

3-Tier CI/CD Strategy

Test Profiles (DIA only)

Version Override Profiles (for merge matrix)

CI Workflows

When to Run Tests Locally

Container Authentication

Development Conventions

Branch Strategy

Naming Conventions

Channel Names

Process/Module Names

Resource Labels

DIA-NN Module Labels

Adding a New DIA-NN Feature

Code Style

Troubleshooting

Pre-commit Issues

Testing Issues

Nextflow Issues

Quick Reference

Essential Commands

File Locations