Skip to content

Commit f9f21fb

Browse files
author
Shane Wall
committed
Updated docs and changelog for the Time Machine pivot and dogfooding CI:
Archived legacy GitHub workflows to docs/archive/v1-legacy/workflows/ and added new self-hosted CI pipeline at .github/workflows/ci.yml; removed scripts/prefetch-btf.sh. Refreshed public docs: README now highlights the dogfooding CI flow; docs index now points to pivot/architecture/specs and notes archives; release notes add v2.0 Time Machine highlights (Pebble, journal/worker, record/export, dogfood CI, archived BoltDB era).
1 parent ebee1bb commit f9f21fb

7 files changed

Lines changed: 195 additions & 106 deletions

File tree

‎.github/workflows/ci.yml‎

Lines changed: 97 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -1,29 +1,108 @@
11
name: CI
2-
on: [push, pull_request]
3-
permissions:
4-
contents: read
5-
packages: write
2+
3+
on:
4+
push:
5+
branches: [ main ]
6+
tags: [ 'v*' ]
7+
pull_request:
8+
branches: [ main ]
69

710
jobs:
8-
build-test:
11+
# 1. Standard Unit Tests & Linting
12+
test:
13+
name: Unit Tests
14+
runs-on: ubuntu-latest
15+
steps:
16+
- uses: actions/checkout@v4
17+
18+
- name: Set up Go
19+
uses: actions/setup-go@v5
20+
with:
21+
go-version: '1.23'
22+
23+
- name: Install dependencies
24+
run: go mod download
25+
26+
- name: Verify Code Formatting
27+
run: if [ -n "$(gofmt -l .)" ]; then echo "Go code is not formatted"; exit 1; fi
28+
29+
- name: Run Unit Tests
30+
# We run tests with race detection to catch concurrency bugs in the new Pebble implementation
31+
run: go test -race -v ./...
32+
33+
# 2. Build Verification (Cross-Platform)
34+
build:
35+
name: Build Binary
936
runs-on: ubuntu-latest
1037
strategy:
1138
matrix:
12-
go-version: ['1.23']
13-
platform: [amd64, arm64]
39+
include:
40+
- os: linux
41+
arch: amd64
42+
- os: linux
43+
arch: arm64
44+
- os: darwin # Verify macOS build (fsnotify fallback)
45+
arch: arm64
46+
steps:
47+
- uses: actions/checkout@v4
48+
- uses: actions/setup-go@v5
49+
with:
50+
go-version: '1.23'
51+
52+
- name: Build
53+
env:
54+
GOOS: ${{ matrix.os }}
55+
GOARCH: ${{ matrix.arch }}
56+
run: |
57+
mkdir -p bin
58+
go build -o bin/diffkeeper-${{ matrix.os }}-${{ matrix.arch }} .
59+
60+
# 3. The "Dogfood" Test (Functional Verification)
61+
# This proves the "Time Machine" feature actually works on Linux (with eBPF/Pebble)
62+
functional-test:
63+
name: Functional Test (The Time Machine)
64+
runs-on: ubuntu-latest
65+
needs: [build]
1466
steps:
1567
- uses: actions/checkout@v4
1668
- uses: actions/setup-go@v5
1769
with:
18-
go-version: ${{ matrix.go-version }}
19-
- uses: docker/setup-buildx-action@v3
20-
id: buildx
21-
- run: make build-ebpf
22-
- run: make build
23-
- run: go test -tags integration -v -timeout 30m ./...
24-
- name: Demo smoke test
25-
shell: bash
70+
go-version: '1.23'
71+
72+
- name: Build DiffKeeper
73+
run: go build -o diffkeeper .
74+
75+
- name: Create Flaky Script
76+
# This simulates the user scenario: A script that changes a file over time
77+
run: |
78+
cat <<'EOF' > flaky.sh
79+
#!/bin/bash
80+
echo "STATE_1: START" > status.txt
81+
sleep 1
82+
echo "STATE_2: MIDDLE" > status.txt
83+
sleep 1
84+
echo "STATE_3: END" > status.txt
85+
EOF
86+
chmod +x flaky.sh
87+
88+
- name: Record Session
89+
# We run as sudo to allow eBPF to attach to the kernel
90+
run: |
91+
sudo ./diffkeeper record --state-dir=/tmp/trace -- ./flaky.sh
92+
93+
- name: Verify Export (Time Travel)
2694
run: |
27-
set -euo pipefail
28-
cd demo/postgres-survive-kill9
29-
./ci-smoke.sh
95+
# 1. Restore state at ~0.5s (Should be STATE_1)
96+
sudo ./diffkeeper export --state-dir=/tmp/trace --out=/tmp/restore_1 --time="500ms"
97+
98+
# 2. Restore state at ~1.5s (Should be STATE_2)
99+
sudo ./diffkeeper export --state-dir=/tmp/trace --out=/tmp/restore_2 --time="1500ms"
100+
101+
# 3. Assertions
102+
echo "Checking State 1..."
103+
grep "STATE_1: START" /tmp/restore_1/status.txt
104+
105+
echo "Checking State 2..."
106+
grep "STATE_2: MIDDLE" /tmp/restore_2/status.txt
107+
108+
echo "✅ Time Travel Successful!"

‎README.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -41,5 +41,9 @@ Now `cd ./debug_fs` and explore the files exactly as they existed at that moment
4141
- **Storage:** Pebble (LSM Tree) for high-speed ingestion.
4242
- **Diffing:** bsdiff (Binary patches) for efficient storage.
4343

44+
## CI / Dogfooding
45+
- GitHub Actions (`.github/workflows/ci.yml`) runs unit/race tests, cross-platform builds, and a functional "time machine" test that records a flaky script and verifies exports at multiple timestamps.
46+
- The BoltDB era workflows are archived under `docs/archive/v1-legacy/workflows/`.
47+
4448
## Getting Started
4549
See the [Quickstart](docs/quickstart.md) to record your first trace.

‎docs/README.md‎

Lines changed: 9 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -3,24 +3,22 @@
33
Quick links to the most useful docs, grouped by purpose.
44

55
## Start Here
6-
- [Quickstart](quickstart.md): run the 60-second demo or wrap your own app.
7-
- [Architecture](architecture.md): capture/replay flow and storage layout.
6+
- [Quickstart](quickstart.md): record/export a flaky script with the Time Machine workflow.
7+
- [Architecture](architecture.md): Firehose → worker → Pebble prefixes (`l:/c:/m:`).
8+
- [Genesis & Pivot](GENESIS_AND_PIVOT.md): why we moved from "stateful containers" to "flight recorder".
89
- [Main README](../README.md): overview and top-level entry points.
910

11+
## Specs & Design
12+
- [PIVOT_SPEC_V2](specs/PIVOT_SPEC_V2.md): storage engine swap, journal pipeline, CLI expectations.
13+
- [GENESIS](GENESIS.md): snapshot of the pivot decision.
14+
1015
## Deploy & Operate
1116
- Kubernetes: [`k8s/README.md`](../k8s/README.md), smoke test (`k8s/SMOKE_TEST.md`), troubleshooting (`k8s/TROUBLESHOOTING.md`), and [test guide](guides/kubernetes-testing.md).
1217
- eBPF/auto-injection: [ebpf-guide.md](ebpf-guide.md), [auto-injection.md](auto-injection.md), [supported-kernels.md](supported-kernels.md), [btf-core-guide.md](btf-core-guide.md).
1318
- Reference: [project structure](reference/project-structure.md), [ebpf-dev-setup.md](ebpf-dev-setup.md), [patents](patents.md).
1419

15-
## History & Planning
16-
- Roadmap and implementation notes live under `docs/history/`:
17-
- [implementation-checklist.md](history/implementation-checklist.md)
18-
- [implementation-plan.md](history/implementation-plan.md)
19-
- [implementation-summary.md](history/implementation-summary.md)
20-
- [phase3-complete.md](history/phase3-complete.md)
21-
- [phase4-test-results.md](history/phase4-test-results.md)
22-
- [v1.0-implementation-complete.md](history/v1.0-implementation-complete.md)
23-
- [v1.0-final-complete.md](history/v1.0-final-complete.md)
20+
## History & Archives
21+
- v1 BoltDB-era documents and demos live under `docs/archive/v1-legacy/` (architecture, history, workflows, and demos).
2422

2523
## Releases
2624
- [Release notes](releases/release-notes.md)

‎docs/archive/v1-legacy/workflows/ci-legacy.yml‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
name: CI
2+
on: [push, pull_request]
3+
permissions:
4+
contents: read
5+
packages: write
6+
7+
jobs:
8+
build-test:
9+
runs-on: ubuntu-latest
10+
strategy:
11+
matrix:
12+
go-version: ['1.23']
13+
platform: [amd64, arm64]
14+
steps:
15+
- uses: actions/checkout@v4
16+
- uses: actions/setup-go@v5
17+
with:
18+
go-version: ${{ matrix.go-version }}
19+
- uses: docker/setup-buildx-action@v3
20+
id: buildx
21+
- run: make build-ebpf
22+
- run: make build
23+
- run: go test -tags integration -v -timeout 30m ./...
24+
- name: Demo smoke test
25+
shell: bash
26+
run: |
27+
set -euo pipefail
28+
cd demo/postgres-survive-kill9
29+
./ci-smoke.sh

.github/workflows/release-windows.yml renamed to docs/archive/v1-legacy/workflows/release-windows-legacy.yml

File renamed without changes.

0 commit comments

Comments
 (0)