Skip to content
BranchesTags

Repository files navigation

Robust Literature Review Pipeline

GitHub stars Last Commit Python License: MIT Render & Release

From topic to submission-ready manuscript in one command. This pipeline produces a complete, publication-grade systematic literature review article — not a skeleton or reference list, but a real academic manuscript with narrative synthesis, specific clinical data, and full PRISMA 2020 compliance.

🔬 Live example — claim appraisal

「4+2R 代謝飲食法」實證評析 → — a systematic-review-grade verdict on a popular diet, built end-to-end by /claim-appraise: 9 claims · 1,056 records → 165 Q1/≥2016/CrossRef-verified studies (97 cited) · GRADE per claim · /argdown-audited, bilingual 專業／民眾 dual-audience.

4+2R 代謝飲食法 evidence appraisal

Why "Robust"?

Most automated review tools stop at collecting references. This pipeline goes all the way to a submission-ready manuscript with built-in quality assurance:

What we deliver How we ensure quality
10,000+ word narrative review 8 parallel writing agents, each with domain-specific context
Specific clinical data (dosing, thresholds, p-values) Haiku subagent structured extraction from full abstracts
Balanced topic coverage Subtopic-aware article selection across 16 categories
Every DOI verified doi.org handle API validation (100% pass rate)
PRISMA 2020 flow diagram TikZ-rendered professional figure
PRISMA 2020 checklist 27-item self-audit with automated gap detection
Auto-repair of gaps Audit loop dispatches repair agents until all items pass
Cover letter to editor Ready-to-submit letter highlighting significance
AMA-formatted citations CSL-driven superscript numbering
PDF + DOCX + BibTeX Quarto multi-format rendering
GitHub Release artifacts Automated CI/CD on every push

Two Modes

Mode Input Output Skill
Topic review a research topic submission-ready manuscript (PDF/DOCX) /lit-review
Claim appraisal a real-world claim a verdict per sub-question, as a published web report /claim-appraise

Claim Appraisal — adjudicate a claim, don't just review a topic

/claim-appraise "<claim>" turns one real-world claim (a popular diet, a supplement, a health product) into a defensible, peer-checkable evidence appraisal:

  1. Decompose the claim into PICO sub-questions (efficacy + safety + over-extended claims).
  2. Per-question systematic review — Scopus + PubMed + Embase, ≥ 2016, Q1 journals only, DOI-validated, and every cited work CrossRef-verified to exist (an anti-hallucination gate — no fabricated citations).
  3. GRADE the certainty of each sub-question (recomputed deterministically), cross-checked against OpenEvidence.
  4. /argdown-audit the overall verdict logic so the conclusion provably follows.
  5. Ship a verdict-style site (bilingual, 專業/民眾 dual-audience) to Cloudflare Pages, with the claim's primary-source URL attached so peers can confirm no cherry-picking (斷章取義).

Live worked example — the "4+2R 代謝飲食法" diet: https://verdict-4plus2r-sr.pages.dev/ 9 sub-claims appraised over 1,056 records → 165 Q1/≥2016/CrossRef-verified studies (97 cited). Verdicts range from 部分成立 (high-protein, meal-order) to 與證據牴觸 (the "重設代謝定點/永不復胖" claim). Reproduce it with scripts/run_4plus2r.py + scripts/build_4plus2r_site.py.

Final Deliverables

Every run produces a complete submission package:

File Purpose
literature_review.pdf The manuscript — ready for journal submission
literature_review.docx Editable Word version for co-author revisions
cover_letter.qmd Cover letter to the editor (auto-generated)
references.bib Validated BibTeX with all DOIs resolved
sections/09-prisma-checklist.qmd Completed PRISMA 2020 checklist (27 items)

Pipeline: Topic to Manuscript

"Hemophagocytic lymphohistiocytosis in adults"
                    |
                    v
    +-----------+----------+-----------+
    |           |          |           |
 Scopus      PubMed     Embase     (parallel)
    |           |          |
    +-----------+----------+
                |
          897 records
                |
          Deduplicate ──> 783 unique
                |
          Quality filter (CiteScore >= 3.0)
                |
          DOI validation (100% via doi.org)
                |
          Fetch full abstracts (PubMed API)
                |
          Balanced subtopic selection ──> 50 articles
                |
          Haiku structured extraction (5 agents)
                |
          Zotero export
                |
    +-----------+--+--+--+--+--+--+--+-----------+
    |  abstract  intro  methods  pathogenesis  ...  |
    |          8 parallel writing agents             |
    +------------------------------------------------+
                |
          PRISMA 2020 audit (27 items)
                |
           pass? ──no──> repair agents ──> re-audit
                |
               yes
                |
          Generate cover letter
                |
          Quarto render (PDF + DOCX)
                |
          GitHub Release

The PRISMA 2020 Audit Loop

After all sections are written, the pipeline automatically:

  1. Audits the manuscript against all 27 PRISMA 2020 checklist items (keyword matching or LLM-as-judge via haiku subagents)
  2. Identifies gaps — items that are missing or partially addressed
  3. Generates targeted fix instructions for each section file that needs improvement
  4. Dispatches repair agents that add only the missing content (no rewrites)
  5. Re-audits until all items pass (max 2 iterations)
from litreview.pipeline.prisma_audit import audit_manuscript, format_audit_report

results = audit_manuscript(Path("output/sections"))
print(format_audit_report(results))
# Score: 35/36 passed (0 failed, 0 partial, 1 N/A)
# All PRISMA 2020 items are addressed. The manuscript is ready for submission.

Human-in-the-Loop Mode

Add --hitl to pause at 9 critical decision points where human judgment is irreplaceable:

/lit-review --hitl "your research topic"
Checkpoint When Why human needed
CP1 Search Strategy After query generation Wrong query = wrong review. Domain expert knows the right terms.
CP2 Borderline Articles After filtering Articles near the threshold need domain judgment to include/exclude.
CP3 Final Article Set After selection Expert may know landmark papers the machine missed.
CP4 Thematic Grouping Before writing How findings are grouped shapes the narrative arc.
CP5 Key Claims After writing Highest error cost. LLMs can hallucinate dosing, p-values, survival rates.
CP6 PRISMA Audit After audit Some items may be legitimately N/A for the review type.
CP7 Cover Letter Before render Target journal selection affects framing.
CP8 Final Preview Before render Last chance to catch major issues.
CP9 Publish Decision Before push Public release requires conscious consent.

Each checkpoint presents multiple-choice options (not open-ended questions), has a default for auto-mode, and logs decisions to checkpoint_log.json for reproducibility.

Without --hitl, the pipeline runs fully automated using defaults.

Quick Start

# Install
uv venv && source .venv/bin/activate && uv pip install -e "."

# Configure API keys
cp .env.example .env
# Edit .env with your API keys

# Run a literature review
lit-review review "your research topic" --target 50 --min-citescore 3.0

# Validate DOIs in existing BibTeX
lit-review validate output/references.bib

# Check API configuration
lit-review check-config

API Keys Required

Service Purpose Get Key
Scopus Article search + CiteScore/SJR metrics Elsevier Developer Portal
PubMed Biomedical literature search NCBI API Key
Embase Medical literature (via Elsevier) Same as Scopus
Unpaywall DOI validation + open access links Unpaywall (just email)
Zotero Reference management export Zotero Settings

Modular Section Architecture

The review is written in parallel using Quarto {{< include >}} syntax:

output/
  literature_review.qmd              # Main file with includes
  cover_letter.qmd                   # Letter to the editor
  references.bib                     # BibTeX references
  prisma-flow-diagram/               # PRISMA TikZ package
  sections/
    00-abstract.qmd                  # Structured abstract (~300 words)
    01-introduction.qmd              # Clinical significance, history, objectives
    02-methods.qmd                   # PRISMA flow, search strategy, quality assessment
    03-pathogenesis.qmd              # IFN-gamma axis, cytokines, ferritin, genetics
    04-diagnosis.qmd                 # HLH-2004, HLH-2024, HScore, MAS criteria
    05-etiology.qmd                  # Infections, malignancy, MAS, iatrogenic
    06-treatment.qmd                 # HLH-94 dosing, targeted therapies, HSCT
    07-covid.qmd                     # COVID-19 and HLH paradigm
    08-discussion.qmd                # Synthesis, controversies, future directions
    09-prisma-checklist.qmd          # PRISMA 2020 checklist (27 items, auto-audited)

What's in the Manuscript

This is not a template — it's a real article with clinical granularity:

  • Diagnostic criteria with exact thresholds: HLH-2004 (8 criteria, 5/8 needed), HScore (cutoff 168, 93% sensitivity, 86% specificity), EULAR/ACR MAS (ferritin >684 + 2/4 criteria)
  • Treatment protocols with dosing: etoposide 150 mg/m² twice weekly, dexamethasone 10 mg/m²/day, anakinra 1-2 mg/kg
  • Trial data with sample sizes: ZUMA-1 (n=108), CARTITUDE-1 (n=97), emapalumab (n=34, 63% response, P=0.02)
  • Every organism named: EBV, CMV, HSV-1/2, HHV-6, HHV-8, HIV, SARS-CoV-2, M. tuberculosis, Ehrlichia, Histoplasma, Plasmodium...
  • Distinct entities distinguished: CRS vs IEC-HS vs classical HLH, HLH-2004 vs HLH-2024, MAS vs secondary HLH

GitHub Actions

On push to main, the workflow automatically:

  • Renders Quarto to PDF + DOCX
  • Creates a GitHub Release with all artifacts (PDF, DOCX, BibTeX, QMD)
  • Uploads artifacts for 90-day retention

Claude Code Integration

This project ships Claude Code skills — type / followed by the skill name to invoke a workflow:

Slash command What it does
/lit-review "topic" Full topic→manuscript pipeline (search → filter → write → PRISMA audit → render)
/lit-review --hitl "topic" Same, with 9 human-in-the-loop checkpoints
/brainstorm-topic Brainstorm and refine search terms before running
/claim-appraise "<claim>" Claim appraisal → PICO decomposition → per-question Q1/≥2016/CrossRef-verified SR → GRADE → verdict → bilingual verdict site (see below)

Skills live in .claude/skills/ and are checked into the repo, so anyone who clones it gets the same / commands.

Citing This Project

If you use this pipeline in your research, please cite:

BibTeX

@software{lin2026robustlitreview,
  author       = {Lin, Hsieh-Ting},
  title        = {Robust Literature Review Pipeline: Automated Systematic Review with Multi-Database Search, DOI Validation, and Publication-Ready Output},
  year         = {2026},
  publisher    = {GitHub},
  url          = {https://github.com/htlin222/robust-lit-review},
  version      = {1.0.0}
}

APA

Lin, H.-T. (2026). Robust Literature Review Pipeline: Automated systematic review with multi-database search, DOI validation, and publication-ready output (Version 1.0.0) [Computer software]. GitHub. https://github.com/htlin222/robust-lit-review

AMA

Lin HT. Robust Literature Review Pipeline: Automated Systematic Review with Multi-Database Search, DOI Validation, and Publication-Ready Output. Version 1.0.0. GitHub; 2026. Accessed March 25, 2026. https://github.com/htlin222/robust-lit-review

Changelog

See CHANGELOG.md for version history.

License

MIT

About

Automated systematic literature review pipeline generating publication-ready manuscripts from Scopus/PubMed/Embase with PRISMA compliance, GRADE evidence synthesis, and DOI validation for clinical research.

Topics

academic-publishingclinical-researchevidence-based-medicineevidence-synthesisgrade-assessmentliterature-reviewmeta-analysisprismapubmedpythonquartoresearch-automationresearch-manuscriptscopussystematic-review

Resources

Readme
MIT license
Activity

Stars

50 stars

Watchers

0 watching

Forks

21 forks
Report repository

Releases

Packages

Contributors

Languages