Make the setup even easier

Author: andraghetti

.dockerignore

@@ -1,7 +1,6 @@
 # Exclude folders that would be mounted
-fastq/
+data/
 runs/
-refs/
 
 # Exclude git and system files
 .git/

.gitignore

@@ -1,9 +1,5 @@
 .DS_Store
-/fastq/*
+/data/*
 /runs/*
-#/programs/**
-/refs/*
-!/refs/empty.vcf
-!/refs/empty.vcf.gz
 ~*
 .Rapp.history

Dockerfile

@@ -53,9 +53,12 @@ RUN R --slave -e "library(ggplot2); library(reshape2); cat('R packages loaded su
 # Set working directory
 WORKDIR /app
 
-# Copy all files (excluding output and fastq directories via .dockerignore)
+# Copy all files (excluding output and data directories via .dockerignore)
 COPY . /app/
 
+# Create runs directory for output
+RUN mkdir -p /app/runs /app/refs
+
 # Install BWA 0.7.19 (latest bug-fixed version) with architecture-specific optimizations
 RUN echo "Installing BWA 0.7.19..." && \
     wget https://github.com/lh3/bwa/archive/refs/tags/v0.7.19.tar.gz && \
@@ -99,5 +102,5 @@ RUN echo "Setting up snpEff..." && \
 # Make scripts executable
 RUN chmod +x ./scripts/simple.sh
 
-# Default command
-CMD ["./scripts/simple.sh"]
+# Set entrypoint
+ENTRYPOINT ["./scripts/simple.sh"]

README.md

@@ -17,45 +17,99 @@ Simple is a bioinformatics pipeline for mapping EMS-induced point mutations usin
 ### Step 1: Prepare Your Data
 
 > [!IMPORTANT]
-> **File Naming Convention**
+> **Create a Working Directory**
 > 
-> Your FastQ files **MUST** follow this exact naming pattern in the `fastq/` directory:
+> **You must create a folder containing your FastQ files before running the pipeline.** The pipeline will create all output in a timestamped subdirectory within this folder.
+> If you have cloned this repository, just use the `data` and `runs` directories.
+> Otherwise create them inside a clean directory.
 
 ### Required File Names:
+The pipeline automatically detects files based on flexible patterns. Files must contain:
+- **"mut"** for mutant bulk samples
+- **"wt"** for wild-type bulk samples  
+- **"R1"** or **"R2"** to indicate read direction
+- **Any extension**: `.fastq`, `.fq`, `.fastq.gz`, `.fq.gz`, etc.
+
+### ✅ Setup Example:
+```bash
+# 1. Create your analysis folder
+mkdir my_mutant_analysis
+cd my_mutant_analysis
+
+# 2. Create data folder for your FastQ files
+mkdir -p data runs
+
+# 3. Copy your FastQ files to the data folder
+# Your folder should now look like this:
+ls -la data/
+# mut.R1.fq.gz    mut.R2.fq.gz    wt.R1.fq.gz    wt.R2.fq.gz
+```
+
+### ✅ Supported Examples:
+```
+my_analysis/
+├── data/
+│   ├── mut.R1.fq.gz          # Mutant bulk, read 1 (compressed)
+│   ├── mut.R2.fq.gz          # Mutant bulk, read 2 (compressed)
+│   ├── wt.R1.fastq           # Wild-type bulk, read 1 (uncompressed)
+│   └── wt.R2.fastq           # Wild-type bulk, read 2 (uncompressed)
+└── runs/                     # Created automatically by pipeline
 ```
-fastq/
-├── mut.R1.fastq    # Mutant bulk, read 1
-├── mut.R2.fastq    # Mutant bulk, read 2 (paired-end)
-├── wt.R1.fastq     # Wild-type bulk, read 1
-└── wt.R2.fastq     # Wild-type bulk, read 2 (paired-end)
+
+### ✅ Alternative Naming (with line prefix):
+```
+my_analysis/
+├── data/
+│   ├── root_mutant.mut.R1.fq.gz
+│   ├── root_mutant.mut.R2.fq.gz
+│   ├── root_mutant.wt.R1.fq.gz
+│   └── root_mutant.wt.R2.fq.gz
+└── runs/                     # Created automatically by pipeline
 ```
 
-### Alternative Naming (with line prefix):
+### ✅ More Examples:
 ```
-fastq/
-├── YOUR_LINE_NAME.mut.R1.fastq
-├── YOUR_LINE_NAME.mut.R2.fastq
-├── YOUR_LINE_NAME.wt.R1.fastq
-└── YOUR_LINE_NAME.wt.R2.fastq
+my_analysis/
+├── data/
+│   ├── sample_mut.R1.fastq.gz
+│   ├── sample_mut.R2.fastq.gz
+│   ├── sample_wt.R1.fastq.gz
+│   └── sample_wt.R2.fastq.gz
+└── runs/                     # Created automatically by pipeline
+```
+
+### 📁 Repository Structure (for developers):
+```
+simple-fork/
+├── data/                 # Example data folder (rename to your analysis name)
+│   ├── mut.R1.fq.gz
+│   ├── mut.R2.fq.gz
+│   ├── wt.R1.fq.gz
+│   └── wt.R2.fq.gz
+├── scripts/              # Pipeline scripts
+├── programs/             # Bioinformatics tools
+└── Dockerfile
 ```
 
 ### ⚠️ Common Mistakes to Avoid:
-- ❌ `mutant.fastq` - Wrong! Must be `mut.R1.fastq`
-- ❌ `wildtype.fastq` - Wrong! Must be `wt.R1.fastq`
-- ❌ `mut_1.fastq` - Wrong! Must be `mut.R1.fastq`
-- ❌ `wt_1.fastq` - Wrong! Must be `wt.R1.fastq`
-- ❌ Missing `.R1` or `.R2` - Wrong! Required for paired-end
+- ❌ `mutant.fastq` - Missing "R1" or "R2" identifier
+- ❌ `wildtype.fastq` - Missing "R1" or "R2" identifier  
+- ❌ `mut_1.fastq` - Should be `mut.R1.fastq`
+- ❌ `wt_1.fastq` - Should be `wt.R1.fastq`
+- ❌ Files without "mut" or "wt" in the name
+- ❌ Files without "R1" or "R2" in the name
 
 ### ✅ Correct Examples:
-- `mut.R1.fastq`, `mut.R2.fastq`, `wt.R1.fastq`, `wt.R2.fastq`
+- `mut.R1.fq.gz`, `mut.R2.fq.gz`, `wt.R1.fq.gz`, `wt.R2.fq.gz`
 - `root_mutant.mut.R1.fastq`, `root_mutant.mut.R2.fastq`, `root_mutant.wt.R1.fastq`, `root_mutant.wt.R2.fastq`
+- `sample_mut.R1.fq`, `sample_mut.R2.fq`, `sample_wt.R1.fq`, `sample_wt.R2.fq`
 
 ### Step 2: Check Available Species
-Before configuring, check what species are available in `scripts/data_base.txt`:
+Before configuring, check what species are available:
 
 ```bash
-# View available species (first column)
-cat scripts/data_base.txt | grep -v "^#" | awk '{print $1}'
+# View available species using Docker
+docker run --rm ghcr.io/andraghetti/simple cat /app/scripts/data_base.txt | grep -v "^#" | awk '{print $1}'
 ```
 
 **Available species include:**
@@ -69,49 +123,52 @@ cat scripts/data_base.txt | grep -v "^#" | awk '{print $1}'
 - `Saccharomyces_cerevisiae`
 
 ### Step 3: Configure the Analysis
-Edit `scripts/simple_variables.sh` to set your parameters:
-
-```bash
-# Change the line name (used as prefix for output files, default is EMS)
-line=YOUR_LINE_NAME
+The pipeline accepts command line arguments that override the default values. You can customize the analysis by passing parameters to the Docker container:
 
-# Set mutation type
-mutation=recessive  # or "dominant"
-
-# Set species (must match entries in scripts/data_base.txt in organism column)
-my_species=Arabidopsis_thaliana
-
-# CPU cores configuration
-cpu_cores="auto"  # Use "auto" for all available cores, or specify a number (e.g., 4)
-
-# Java memory allocation
-java_memory="8g"  # Set RAM for Java tools (e.g., "8g", "16g", "4g")
-```
+**Available Parameters:**
+- `--line LINE_NAME`: Line name (default: EMS)
+- `--mutation TYPE`: Mutation type: recessive or dominant (default: recessive)
+- `--species SPECIES`: Species name (default: Arabidopsis_thaliana)
+- `--cpu-cores CORES`: CPU cores: auto or number (default: auto)
+- `--memory MEMORY`: Java memory allocation (default: auto)
 
 ### Step 4: Run the Pipeline
 
 #### Option A: Use Pre-built Docker Image (Recommended)
 ```bash
 # Pull the pre-built image from GitHub Container Registry
-# No login required - images are publicly accessible
-# Docker automatically detects your platform (AMD64/ARM64) and pulls the correct image
-docker pull ghcr.io/andraghetti/Simple:latest
+docker pull ghcr.io/andraghetti/simple:latest
 
-# Run the pipeline
+# Run the pipeline with default settings
 docker run --rm \
-  -v $(pwd)/fastq:/app/fastq:ro \
+  -v $(pwd)/data:/app/data \
   -v $(pwd)/runs:/app/runs \
-  ghcr.io/andraghetti/Simple:latest
+  ghcr.io/andraghetti/simple:latest
+```
+
+Alternative example with all the configurable options:
+
+```bash
+# Run the pipeline with custom parameters
+docker run --rm \
+  -v $(pwd)/data:/app/data \
+  -v $(pwd)/runs:/app/runs \
+  ghcr.io/andraghetti/simple:latest \
+  --line root_mutant \
+  --mutation recessive \
+  --species Arabidopsis_thaliana \
+  --cpu-cores 8 \
+  --java-memory 16g
 ```
 
 #### Option B: Build Docker Image Locally
 ```bash
 # Build with explicit platform specification (works with regular docker build)
 docker build -t simple-pipeline .
 
-# Run the pipeline
+# Run the pipeline with default settings
 docker run --rm \
-  -v $(pwd)/fastq:/app/fastq:ro \
+  -v $(pwd)/data:/app/data \
   -v $(pwd)/runs:/app/runs \
   simple-pipeline
 ```
@@ -121,20 +178,10 @@ After completion, check the `runs/` directory for your results:
 - `runs/run-YYYYMMDD_HHMMSS/output/YOUR_LINE_NAME.candidates.txt` - Candidate mutations
 - `runs/run-YYYYMMDD_HHMMSS/output/YOUR_LINE_NAME.allSNPs.txt` - All SNPs for plotting
 - `runs/run-YYYYMMDD_HHMMSS/output/YOUR_LINE_NAME.Rplot_*.pdf` - Manhattan plots
+- `runs/run-YYYYMMDD_HHMMSS/output/snpEff_summary.html` - snpEff annotation summary
+- `runs/run-YYYYMMDD_HHMMSS/output/snpEff_genes.txt` - snpEff gene annotations
 - `runs/run-YYYYMMDD_HHMMSS/output/log.txt` - Complete execution log
-- `runs/run-YYYYMMDD_HHMMSS/archive/` - All intermediate files
-- `runs/run-YYYYMMDD_HHMMSS/refs/` - Reference files used
-- `runs/run-YYYYMMDD_HHMMSS/fastq/` - Input FASTQ files
 
-## Features
-
-- **Multi-Architecture Support**: Optimized for both AMD64 and ARM64 (Apple Silicon)
-- **Dockerized Environment**: All dependencies pre-installed and compiled
-- **Self-Contained Runs**: Each run creates a timestamped directory with all data
-- **Pre-compiled Tools**: Latest versions of BWA, samtools, Picard, and snpEff
-- **Multi-Core Performance**: Automatically detects and uses all available CPU cores
-- **High Memory Allocation**: 8GB RAM (configurable) allocated for Java tools
-- **Public Docker Images**: No build time required, freely accessible from GitHub Container Registry
 
 ### Pipeline Steps
 1. **Reference Preparation**: Downloads and indexes reference genome
@@ -147,27 +194,43 @@ After completion, check the `runs/` directory for your results:
 ## Example Analysis
 
 ### Root Development Mutant Analysis
-1. **Prepare files**: `root_mutant.mut.R1.fastq`, `root_mutant.mut.R2.fastq`, `root_mutant.wt.R1.fastq`, `root_mutant.wt.R2.fastq`
-2. **Configure**: Set `line=root_mutant`, `mutation=recessive`, `my_species=Arabidopsis_thaliana`
-3. **Run**: Execute the Docker pipeline
-4. **Results**: Check `root_mutant.candidates.txt` for candidate genes affecting root development
-
-## Supported Species
-
-The pipeline supports various species listed in `scripts/data_base.txt`. Common examples:
-- Arabidopsis_thaliana
-- Oryza_sativa_Japonica
-- Zea_mays
-- And many more...
+1. **Create analysis folder and add files**:
+   ```bash
+   # Create your analysis folder
+   mkdir root_analysis
+   cd root_analysis
+   
+   # Add your FastQ files (copy or download them here)
+   # Files: root_mutant.mut.R1.fq.gz, root_mutant.mut.R2.fq.gz
+   #        root_mutant.wt.R1.fq.gz, root_mutant.wt.R2.fq.gz
+   ```
+
+2. **Run the pipeline**:
+   ```bash
+   docker run --rm \
+     -v $(pwd)/data:/app/data \
+     -v $(pwd)/runs:/app/runs \
+     ghcr.io/andraghetti/simple:latest \
+     --line root_mutant \
+     --mutation recessive \
+     --species Arabidopsis_thaliana
+   ```
+
+3. **Check results**: Look in `runs/run-YYYYMMDD_HHMMSS/output/root_mutant.candidates.txt` for candidate genes affecting root development
 
 ## Troubleshooting
 
 ### Common Issues
 
-1. **Pipeline fails immediately**: Check file naming convention in `fastq/` directory
-2. **Out of memory errors**: Reduce `java_memory` in `simple_variables.sh`
-3. **Slow performance**: Ensure `cpu_cores="auto"` for maximum performance
-4. **Missing reference**: Check `my_species` matches entries in `data_base.txt`
+1. **Pipeline fails immediately**: Check file naming convention in your analysis directory
+   - Files must contain "mut" or "wt" AND "R1" or "R2"
+   - Examples: `mut.R1.fq.gz`, `wt.R1.fastq`, `line_mut.R1.fq.gz`
+2. **Out of memory errors**: Reduce `--memory` parameter (e.g., use `--memory 8g`)
+3. **Slow performance**: Ensure `--cpu-cores auto` for maximum performance
+4. **Missing reference**: Check `--species` matches entries in `data_base.txt`
+5. **Invalid arguments**: Use `--help` to see available options
+6. **No files found**: Verify files are in the current directory with correct naming pattern
+7. **Permission errors**: Ensure Docker has read/write access to your analysis directory
 
 ### Getting Help
 
@@ -181,4 +244,4 @@ This project is licensed under the MIT License - see the [LICENSE](LICENSE) file
 
 ## Citation
 
-If you use this pipeline in your research, please cite the original Simple paper and acknowledge this Dockerized version.
+If you use this pipeline in your research, please cite the original Simple paper and acknowledge this optimized version.