Skip to content

About

Crawler for collecting information about CVE patch status per Linux distribution

Resources

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

CVE Distro Tracker

A security tool that aggregates CVE (Common Vulnerabilities and Exposures) fix status information from multiple Linux distribution security trackers and presents it through a searchable web interface.

Architecture Overview

┌─────────────────────────────────────────────────────────────────────────┐
│                           GitHub Actions                                │
│  ┌─────────────────────────────────────────────────────────────────┐    │
│  │  Scheduled Job (Daily at 4 AM & 6 AM UTC)                       │    │
│  │  1. Checkout repository                                         │    │
│  │  2. Run fetcher.main                                            │    │
│  │  3. Commit data/ changes → push to repo                         │    │
│  └─────────────────────────────────────────────────────────────────┘    │
└─────────────────────────────────────────────────────────────────────────┘
                                    │
                                    ▼
┌─────────────────────────────────────────────────────────────────────────┐
│                         Python Fetcher                                  │
│  ┌───────────┐   ┌───────────┐   ┌───────────┐   ┌───────────┐          │
│  │  Alpine   │   │  Debian   │   │  Ubuntu   │   │   RHEL    │          │
│  │  Fetcher  │   │  Fetcher  │   │  Fetcher  │   │  Fetcher  │          │
│  └─────┬─────┘   └─────┬─────┘   └─────┬─────┘   └─────┬─────┘          │
│        │               │               │               │                │
│        └───────────────┴───────┬───────┴───────────────┘                │
│                                ▼                                        │
│                    ┌───────────────────────┐                            │
│                    │    Merge Results      │                            │
│                    │  (merge_results())    │                            │
│                    └───────────┬───────────┘                            │
│                                ▼                                        │
│                    ┌───────────────────────┐                            │
│                    │    Output Writer      │                            │
│                    │  (write_output())     │                            │
│                    └───────────────────────┘                            │
└─────────────────────────────────────────────────────────────────────────┘
                                    │
                                    ▼
┌─────────────────────────────────────────────────────────────────────────┐
│                           data/ Directory                               │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐  ┌────────────┐   │
│  │  index.json  │  │ cve-2026.json│  │ cve-2025.json│  │    ...     │   │
│  └──────────────┘  └──────────────┘  └──────────────┘  └────────────┘   │
└─────────────────────────────────────────────────────────────────────────┘
                                    │
                                    ▼
┌─────────────────────────────────────────────────────────────────────────┐
│                         Static Web Frontend                             │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐                   │
│  │  index.html  │  │   app.js     │  │  styles.css  │                   │
│  │  (Alpine.js) │  │  (fetch API) │  │              │                   │
│  └──────────────┘  └──────────────┘  └──────────────┘                   │
└─────────────────────────────────────────────────────────────────────────┘

External Data Sources

Alpine Linux

Source Type: HTTP JSON API + HTML scraping

Endpoints:

Endpoint Purpose Auth
https://secdb.alpinelinux.org/v{version}/{repo}.json Fixed CVEs from security database None
https://security.alpinelinux.org/recent?page={n} Recent CVEs list (HTML) None
https://security.alpinelinux.org/vuln/{CVE-ID} Individual CVE details (JSON via Accept header) None

Data Retrieved:

  • Package names, versions with security fixes
  • CVE IDs mapped to fixed versions per Alpine release
  • Affected/fixed status from security tracker

Fetcher: AlpineFetcher (fetcher/sources/alpine.py)

Versions Tracked: 3.18, 3.19, 3.20, 3.21, 3.22, 3.23, edge

Debian

Source Type: HTTP JSON API

Endpoint:

Endpoint Purpose Auth
https://security-tracker.debian.org/tracker/data/json Complete CVE database None

Data Retrieved:

  • All CVEs affecting Debian packages
  • Status per release (resolved, open, undetermined, etc.)
  • Fixed versions, urgency levels, package names

Fetcher: DebianFetcher (fetcher/sources/debian.py)

Releases Tracked: trixie, sid, bookworm, bullseye, buster

Data Size: ~50-100 MB JSON file

Ubuntu

Source Type: Git repository (sparse checkout)

Repository: https://git.launchpad.net/ubuntu-cve-tracker

Data Retrieved:

  • CVE files from the active/ directory
  • Status per release extracted from structured text files
  • Priority levels, affected packages, fix versions

Fetcher: UbuntuGitFetcher (fetcher/sources/ubuntu_git.py)

Releases Tracked: trusty, xenial, bionic, focal, jammy, noble, questing, resolute, devel

Git Operations:

# Initial clone (sparse checkout - only fetches metadata)
git clone --depth 1 --filter=blob:none --sparse <repo> <dir>
git -C <dir> sparse-checkout set active

# Updates
git -C <dir> pull --depth=1

File Format (CVE text file):

Candidate: CVE-2026-1234
Priority: high
...
focal_nginx: released (1.18.0-6ubuntu14.11)
jammy_nginx: needed
noble_nginx: not-affected (code not present)

Red Hat Enterprise Linux (RHEL)

Source Type: VEX (Vulnerability Exploitability eXchange) feed

Endpoints:

Endpoint Purpose Auth
https://security.access.redhat.com/data/csaf/v2/vex-feed/index.txt Index of CVE files None
https://security.access.redhat.com/data/csaf/v2/vex-feed/{year}/cve-{id}.json Individual CVE VEX documents None

Data Retrieved:

  • CSAF VEX format documents
  • Product status (fixed, known_affected, known_not_affected)
  • Severity from aggregate_severity field
  • Product tree mapping

Fetcher: RHELVexFetcher (fetcher/sources/rhel_vex.py)

Products Tracked: RHEL 6, 7, 8, 9, 10, OpenShift

Rate Limiting: Fetches in batches of 50 with 0.5s delay between batches

Running Locally

Prerequisites

  • Python 3.9+
  • Git (for Ubuntu fetcher)

Installation

pip install -r requirements.txt

Running the Fetcher

# Fetch all distros (using cache if fresh)
python -m fetcher.main

# Fetch specific distros only
python -m fetcher.main --distros debian ubuntu

# Force fresh fetch (ignore cache)
python -m fetcher.main --no-cache

# Clear cache before fetching
python -m fetcher.main --clear-cache

# Set custom cache age (hours)
python -m fetcher.main --cache-age 12

Running the Web Frontend

# Start local HTTP server
python -m http.server 8000

# Or on Windows (PowerShell helper)
.\distro-cve-checker.ps1

Then open http://localhost:8000 in a browser.

URL Parameters

The web interface supports URL parameters for sharing:

http://localhost:8000/?cve=CVE-2024-1234

File Reference

File Purpose
fetcher/main.py CLI entry point, orchestrates fetching
fetcher/config.py Constants: versions, timeouts, years
fetcher/cache.py Local file caching
fetcher/models.py Pydantic data models
fetcher/output/writer.py JSON output generation
fetcher/sources/base.py Base fetcher class, HTTP utilities
fetcher/sources/alpine.py Alpine Linux fetcher
fetcher/sources/debian.py Debian fetcher
fetcher/sources/ubuntu_git.py Ubuntu fetcher (git-based)
fetcher/sources/rhel_vex.py RHEL VEX feed fetcher
index.html Web frontend HTML
app.js Alpine.js frontend logic
styles.css Frontend styling
data/index.json Generated metadata
data/cve-YYYY.json Generated CVE data
.github/workflows/fetch-cves.yml GitHub Actions daily fetch workflow
amplify.yml AWS Amplify deployment config
buildspec.yml AWS CodeBuild config
deploy.sh Manual deployment helper

About

Crawler for collecting information about CVE patch status per Linux distribution

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Used by

Contributors

Languages