Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -8,4 +8,4 @@ RUN python -m pip install --no-cache-dir --no-cache /opt/PartiNet

LABEL AUTHORS Mihin Perera, Edward Yang, Julie Iskander
LABEL MAINTAINERS Mihin Perera, Edward Yang, Julie Iskander
LABEL VERSION v0.2.0
LABEL VERSION v1.0.1
33 changes: 19 additions & 14 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
# PartiNet 🔬
# PartiNet

![PartiNet RMSD](docs/static/img/partinet_RMSD.png)

PartiNet is a three-stage pipeline for automated particle picking in cryo-EM micrographs, combining advanced denoising with state-of-the-art deep learning detection.

Expand All @@ -9,12 +11,12 @@ Use our pretrained model at [Model Weights](https://huggingface.co/MihinP/PartiN

## Features

- 🧹 Heuristic denoising for improved signal-to-noise ratio
- 🎯 Dynamic deep learning particle detection
- Multi-GPU support for faster processing
- 🔄 Seamless integration with cryoSPARC and RELION workflows
- 📊 Confidence-based particle filtering
- 🖼️ Visual detection validation
- Heuristic denoising for improved signal-to-noise ratio
- Dynamic deep learning particle detection
- Multi-GPU support for faster processing
- Seamless integration with cryoSPARC and RELION workflows
- Confidence-based particle filtering
- Visual detection validation

## Prerequisites

Expand Down Expand Up @@ -49,12 +51,12 @@ docker run --gpus all -v /path/to/your/data:/data \

```
project_directory/
├── motion_corrected/ # 📁 Input micrographs
├── denoised/ # 🧹 Denoised outputs
├── exp/ # 🎯 Detection results
│ ├── labels/ # 📋 Coordinates
│ └── ... # 🖼️ Visualizations
└── partinet_particles.star # Final output
├── motion_corrected/ # Input micrographs
├── denoised/ # Denoised outputs
├── exp/ # Detection results
│ ├── labels/ # Coordinates
│ └── ... # Visualizations
└── partinet_particles.star # Final output
```

## Pipeline Stages
Expand Down Expand Up @@ -94,7 +96,8 @@ partinet star \
- `*.png`: Visualization overlays

3. **STAR File** (`partinet_particles.star`)
- Ready for RELION processing
- Ready for CryoSPARC processing
- Additional commands available for RELION compatible output

## Advanced Usage

Expand Down Expand Up @@ -145,3 +148,5 @@ Citation information will be added upon publication
For issues and questions:
- Open an [Issue](https://github.com/WEHI-ResearchComputing/PartiNet/issues)
- Check existing [Discussions](https://github.com/WEHI-ResearchComputing/PartiNet/discussions)


2 changes: 1 addition & 1 deletion Singularity
Original file line number Diff line number Diff line change
Expand Up @@ -18,4 +18,4 @@ from: python:3.9.19-slim-bookworm
%labels
AUTHORS Mihin Perera, Edward Yang, Julie Iskander
MAINTAINERS Mihin Perera, Edward Yang, Julie Iskander
VERSION v1.0.0
VERSION v1.0.1
12 changes: 6 additions & 6 deletions docs/docs/getting-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,23 +20,23 @@ PartiNet expects and creates the following directory structure:

```
project_directory/
├── motion_corrected/ # 📁 Your input micrographs
├── motion_corrected/ # Your soft-linked input micrographs
│ ├── micrograph1.mrc
│ ├── micrograph2.mrc
│ └── ...
├── denoised/ # 🧹 Created by denoise stage
├── denoised/ # Created by denoise stage
│ ├── micrograph1.mrc
│ ├── micrograph2.mrc
│ └── ...
├── exp/ # 🎯 Created by detect stage
│ ├── labels/ # 📋 Detection coordinates
├── exp/ # Created by detect stage
│ ├── labels/ # Detection coordinates (YOLO format)
│ │ ├── micrograph1.txt
│ │ ├── micrograph2.txt
│ │ └── ...
│ ├── micrograph1.png # 🖼️ Micrographs with detections drawn
│ ├── micrograph1.png # Micrographs with detections drawn
│ ├── micrograph2.
│ └── ...
└── partinet_particles.star # ⭐ Final STAR file (created by star stage)
└── partinet_particles.star # CryoSPARC-style STAR file (created by star stage)
```

**Pipeline Flow:**
Expand Down
9 changes: 3 additions & 6 deletions docs/docs/intro.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,17 +32,14 @@ The final stage prepares particle data for further processing and provides repor
PartiNet is ideal for:
- Identifying rare views
- Picking on heterogeneous datasets
- Reporting on particle populations
- High speed picking

## Next Steps

- **New to PartiNet?** Start with [Installation](installation.md) to get up and running
- **Ready to begin?** Follow our [Getting Started](getting-started.md) guide for your first analysis
- **Need specific details?** Check the individual stage documentation: [Denoise](stages/denoise.md), [Detect](stages/detect.md), [Star](stages/star.md)

<!-- ## Getting Help
## Getting Help

If you encounter issues or need assistance:
- Check the [Troubleshooting](reference/troubleshooting.md) guide
- Review the complete [CLI Reference](reference/cli-reference.md)
- Look at our [Examples](examples/) for common use cases -->
If you encounter issues or need assistance please raise an issue on the GitHub
8 changes: 4 additions & 4 deletions docs/docs/stages/denoise.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,16 +69,16 @@ mkdir motion_corrected
**From CryoSPARC:**
```shell
# Using symbolic links (faster, saves space)
ln -s /path/to/cryosparc/project/job_number/motioncorrected/*_fractions_patch_aligned.mrc motion_corrected/
ln -s /path/to/cryosparc/project/JXXX/motioncorrected/*_fractions_patch_aligned.mrc motion_corrected/

# Using rsync (copies files)
rsync /path/to/cryosparc/project/job_number/motioncorrected/*_fractions_patch_aligned.mrc motion_corrected/
rsync /path/to/cryosparc/project/JXXX/motioncorrected/*_fractions_patch_aligned.mrc motion_corrected/
```

**From RELION:**
```shell
# Link motion-corrected micrographs
ln -s /path/to/relion/project/MotionCorr/job_number/Micrographs/*.mrc motion_corrected/
ln -s /path/to/relion/project/MotionCorr/jobXXX/*.mrc motion_corrected/
```

### 3. Run Denoising
Expand Down Expand Up @@ -166,7 +166,7 @@ partinet denoise \


### Different Output Formats
By default PartiNet outputs denoised images in `png` format. This is necessary for compatibility with the detection architecture. `png` is a lossless compression, however micrographs are normalised from 32 bit depth `mrc` files to 8 bit `png`. `jpg` is also available (eg for making figures) but is not recommended for use due to lossy compression.
By default PartiNet outputs denoised images in `png` format. This is necessary for compatibility with the detection architecture. `png` is a lossless compression, however micrographs are normalised from 32 bit depth `mrc` files to 8 bit `png`. `jpg` is also available (as a legacy format) but is not recommended for use due to lossy compression.

```shell
# JPEG format (smaller file size, lossy compression)
Expand Down
28 changes: 25 additions & 3 deletions docs/docs/stages/star.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,12 +81,32 @@ Choose your confidence threshold based on:
- Downstream processing requirements
- Balance between particle quantity and quality

## Output
## CryoSPARC Output

The command generates a STAR file containing:
The command generates a STAR file compatible with CryoSPARC containing:
- Particle coordinates (X, Y positions)
- Corresponding micrograph paths

## RELION output

Use `--relion` and `--relion-project-dir` to generate RELION-compatible STAR outputs under `<relion_project>/partinet`:

- Micrograph manifest: `<relion_project>/partinet/pick.star`
- Per-micrograph coordinates: `<relion_project>/partinet/movies/<micrograph_basename>.star`

Example:

```bash
partinet star \
--labels /data/partinet_picking/exp/labels \
--images /data/partinet_picking/denoised \
--output /data/partinet_picking/output.star \
--conf 0.2 \
--relion \
--relion-project-dir /data/relion/EMPIAR-10089 \
--mrc-prefix MotionCorr/job003/movies
```


## Next Steps

Expand All @@ -96,8 +116,10 @@ After generating your STAR file, you can:

:::warning

PartiNet Denoise flips micrographs in the y-axis due to matrix transpose operations. If you denoised micrographs in PartiNet ensure that when you import and extract particle coordinates that you toggle `Flip in y` in CryoSPARC or similar in RELION prior to particle extraction.
If you are using PartiNet v1.0.0, the Denoise flips micrographs in the y-axis due to matrix transpose operations. If you denoised micrographs in PartiNet ensure that when you import and extract particle coordinates that you toggle `Flip in y` in CryoSPARC prior to particle extraction. This has been fixed in v1.0.1+. You can check your version with `partinet --help`

:::




4 changes: 2 additions & 2 deletions docs/src/components/HomepageFeatures/index.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ const FeatureList: FeatureItem[] = [
Svg: require('@site/static/img/undraw_success_288d.svg').default,
description: (
<>
PartiNet proides improved identification of rare particle views allowing for comprehensive map reconstruction.
PartiNet provides improved identification of rare particle views allowing for comprehensive map reconstruction.
</>
),
},
Expand All @@ -33,7 +33,7 @@ const FeatureList: FeatureItem[] = [
Svg: require('@site/static/img/undraw_space-exploration_dhu1.svg').default,
description: (
<>
PartiNet provides up to 10× faster inference than existing tools enabling real-time, on-the-fly picking.
PartiNet provides up to 7× faster inference than existing tools enabling real-time, on-the-fly picking.
</>
),
},
Expand Down
10 changes: 8 additions & 2 deletions docs/src/pages/index.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ import useDocusaurusContext from '@docusaurus/useDocusaurusContext';
import Layout from '@theme/Layout';
import HomepageFeatures from '@site/src/components/HomepageFeatures';
import Heading from '@theme/Heading';
import partinetRMSD from '@site/static/img/partinet_RMSD.png';

import styles from './index.module.css';

Expand All @@ -17,6 +18,11 @@ function HomepageHeader() {
{siteConfig.title}
</Heading>
<p className="hero__subtitle">{siteConfig.tagline}</p>
<img
src={partinetRMSD}
alt="PartiNet RMSD"
style={{ maxWidth: '100%', marginTop: '1rem' }}
/>
<div className={styles.buttons}>
<Link
className="button button--secondary button--lg"
Expand All @@ -33,8 +39,8 @@ export default function Home(): ReactNode {
const {siteConfig} = useDocusaurusContext();
return (
<Layout
title={`Hello from ${siteConfig.title}`}
description="Description will go into a meta tag in <head />">
title={`PartiNet: high performance particle picking for cryo-EM`}
description="PartiNet is a high performance particle picking pipeline for cryo-electron microscopy (cryo-EM) that provides improved identification of rare particle views and up to 7× faster inference than existing tools.">
<HomepageHeader />
<main>
<HomepageFeatures />
Expand Down
Binary file added docs/static/img/partinet_RMSD.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
19 changes: 15 additions & 4 deletions partinet/__init__.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import click
import sys, os

__version__ = "1.0.0"
__version__ = "1.0.1"

DYNAMICDET_AVAILABLE_MODELS = ["yolov7", "yolov7x", "yolov7-w6", "yolov7-e6", "yolov7-d6", "yolov7-e6e"]

Expand Down Expand Up @@ -77,12 +77,23 @@ def split(star, images, output, class_id, test_size, split_only):
@main.command()
@click.option("--labels", type=str, required=True, help="Path to the labels directory")
@click.option("--images", type=str, required=True, help="Path to the images directory")
@click.option("--output", type=str, required=True, help="Path to the output STAR file")
@click.option("--output", type=str, required=True, help="Path to the output STAR file (CryoSPARC style)")
@click.option("--conf", type=float, default=0.1, help="Minimum confidence threshold from predictions")
def star(labels, images, output,conf):
@click.option("--relion", is_flag=True, default=False, help="Also generate RELION pick.star + per-micrograph star files")
@click.option("--relion-project-dir", type=str, default=None, help="RELION project root (creates <project>/partinet/pick.star and <project>/partinet/movies/*.star)")
@click.option("--mrc-prefix", type=str, default="", help="Prefix path for micrograph names in RELION pick.star (e.g. MotionCorr/job003/movies)")
def star(labels, images, output, conf, relion, relion_project_dir, mrc_prefix):
click.echo("Generating STAR file...")
import partinet.process_utils.star_file
partinet.process_utils.star_file.main(labels,images,output,conf)
partinet.process_utils.star_file.main(
labels,
images,
output,
conf,
relion=relion,
relion_project_dir=relion_project_dir,
mrc_prefix=mrc_prefix,
)

@main.command()
@click.option("--source", type=str, required=True, help="Path to Raw micrographs")
Expand Down
2 changes: 0 additions & 2 deletions partinet/process_utils/guided_denoiser.py
Original file line number Diff line number Diff line change
Expand Up @@ -167,8 +167,6 @@ def denoise(image_path: str) -> np.ndarray:
# described in https://github.com/WEHI-ResearchComputing/PartiNet/issues/41
image = image.astype(np.float32)

image = image.T
image = np.rot90(image)
normalized_image = standard_scaler(np.array(image))
contrast_enhanced_image = contrast_enhancement(normalized_image)
weiner_filtered_image = wiener_filter(contrast_enhanced_image, kernel, K=30)
Expand Down
Loading
Loading