Skip to content

Commit 46f649f

Browse files
myzieclaude
myzie
and
claude
authored
docs: refresh README and CLAUDE.md after recent worker/store updates (#40)
Reflect the changes from PRs #37-#39: richer Claim fields, HandlerContext with the only-Checkpointer-is-fenced caveat, the runquery subpackage, configurable Postgres schema via WithSchema, atomic DeleteRun, the v0.0.3 single-tenant upgrade carry-forward, and the SQLite coexistence note. Lighter tone throughout; replaced the "Going to production" section with a friendlier "Bring your own storage" framing. Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent 8d39f63 commit 46f649f

2 files changed

Lines changed: 100 additions & 77 deletions

File tree

‎CLAUDE.md‎

Lines changed: 32 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -3,12 +3,13 @@
33
Go library for defining and executing multi-step processes as directed graphs.
44
Module: `github.com/deepnoodle-ai/workflow`
55

6-
Read `llms.txt` for the full API reference.
6+
Read `llms.txt` for the full API reference. The friendly tour lives in
7+
`documentation/` and `README.md`.
78

89
## Scope
910

10-
This is a **pure execution engine**. It runs workflows in-process and provides
11-
interfaces for the things it doesn't own. What it does and doesn't do:
11+
This is a **pure execution engine**. It runs workflows in-process and lets
12+
consumers plug in everything else.
1213

1314
**Does**: Define workflows as step graphs. Execute steps (activities). Branch
1415
and join execution paths. Retry with backoff. Catch and route errors. Checkpoint
@@ -20,12 +21,11 @@ parameter templates via a bundled expression engine
2021
**Does not**: Store workflows, checkpoints, or progress. Queue or schedule work.
2122
Manage distributed workers or leases. Provide a database, API, or UI.
2223

23-
Storage is the consumer's responsibility. The library defines interfaces
24+
Storage is the consumer's problem. The library defines interfaces
2425
(`Checkpointer`, `StepProgressStore`, `ActivityLogger`, `SignalStore`,
25-
`WorkflowRegistry`) and the consumer provides implementations backed by their
26-
own infrastructure (Postgres, Redis, S3, etc.). The built-in
27-
`FileCheckpointer` and `MemoryCheckpointer` exist for development and testing
28-
only.
26+
`WorkflowRegistry`) and the consumer plugs in whatever they like (Postgres,
27+
Redis, S3, etc.). The built-in `FileCheckpointer` and `MemoryCheckpointer` are
28+
for development and testing.
2929

3030
## How Branch Execution Works
3131

@@ -104,20 +104,31 @@ Experimental submodules (separate `go.mod`, not imported by the root module):
104104

105105
- `experimental/worker/` — queue-backed durable worker (claim loop, heartbeat,
106106
reaper, credit reconciler). Defines the `QueueStore` and `HandlerStores`
107-
interfaces. Handlers receive a `*HandlerContext` carrying the `Claim` plus
108-
optional pre-fenced `Checkpointer`, `StepProgressStore`, `ActivityLogger`,
109-
and `SignalStore`. IDs for outbox rows (triggers, webhooks) are generated
110-
worker-side via `Config.IDGenerator`.
107+
interfaces. `Claim` carries `WorkerID`, `ProjectID`, `ParentRunID`,
108+
`InitiatedBy`, and `Metadata`. Handlers receive a `*HandlerContext` carrying
109+
the `Claim` plus optional pre-fenced `Checkpointer`, `StepProgressStore`,
110+
`ActivityLogger`, and `SignalStore` — only `Checkpointer` is lease-fenced;
111+
the others accept `*Claim` for API symmetry. `SignalStore` is shared across
112+
claims and lives on `Config`, not the factory. Outbox row IDs (triggers,
113+
webhooks) are generated worker-side via `Config.IDGenerator`.
114+
- `experimental/worker/runquery/` — backend-neutral run query package
115+
(stdlib-only) that holds `Run`, `RunFilter`, `RunCursor`, `ErrRunNotFound`,
116+
`ErrCannotDeleteRunning`, and the `Store` interface. Dashboards depend on
117+
this package instead of importing a specific store implementation.
111118
- `experimental/store/postgres/` — pgx-backed persistence implementing
112-
`Checkpointer`, `StepProgressStore`, `ActivityLogger`, `CreditStore`, and
113-
`QueueStore`. Exposes a read-side `Run` API (`GetRun`, `ListRuns`,
114-
`CountRuns`, `DeleteRun`) for operational dashboards. Schema namespace is
115-
configurable via `WithSchema(...)`; defaults to `"public"`. `Migrate` issues
116-
`CREATE SCHEMA IF NOT EXISTS` and templates `schema.sql` with the chosen
117-
schema. Schema names are validated as simple SQL identifiers.
118-
- `experimental/store/sqlite/` — database/sql-backed persistence with the same
119-
interface surface. Single-writer, suitable for dev/testing and single-process
120-
deployments.
119+
`Checkpointer`, `StepProgressStore`, `ActivityLogger`, `CreditStore`,
120+
`QueueStore`, and `runquery.Store` (`GetRun`, `ListRuns` with keyset
121+
pagination + metadata JSONB containment, `CountRuns`, `DeleteRun`).
122+
`DeleteRun` is a single atomic statement that refuses running runs.
123+
Schema namespace is configurable via `WithSchema(...)`; defaults to
124+
`"public"`. `Migrate` issues `CREATE SCHEMA IF NOT EXISTS` and templates
125+
`schema.sql` with the chosen schema. Schema names are validated as simple
126+
SQL identifiers. The migration carries forward v0.0.3 single-tenant rows
127+
by rewriting empty-string `org_id`/`initiated_by` to `NULL`.
128+
- `experimental/store/sqlite/` — `database/sql`-backed persistence with the
129+
same interface surface. Single-writer, suitable for dev/testing and
130+
single-process deployments. No schema namespacing escape hatch — consumers
131+
who need coexistence should hand the library a dedicated `*sql.DB`.
121132

122133
## Conventions
123134

‎README.md‎

Lines changed: 68 additions & 56 deletions
Original file line numberDiff line numberDiff line change
@@ -1,31 +1,32 @@
11
# Workflow
22

3-
A Go library for defining and executing multi-step processes as
4-
directed graphs. Conditional branching, parallel execution,
5-
expression-driven templating, durable checkpointing, and
6-
suspend/resume on signals or wall-clock waits.
3+
A Go library for defining and running multi-step processes as
4+
directed graphs. You get conditional branching, parallel execution,
5+
expression-driven templates, durable checkpointing, and the ability
6+
to suspend and resume on signals or wall-clock waits.
77

88
Think of it like a lightweight hybrid of Temporal and AWS Step
9-
Functions, with everything that doesn't belong inside an execution
10-
engine pushed out to interfaces consumers implement.
9+
Functions. The execution engine is all that lives inside; everything
10+
that doesn't belong inside an engine — storage, queues, leasing — is
11+
an interface you implement however you like.
1112

1213
Edge conditions and `${...}` parameter templates are evaluated by
1314
[`github.com/deepnoodle-ai/expr`](https://github.com/deepnoodle-ai/expr),
1415
a small zero-dependency expression evaluator with a Go-like syntax.
15-
It is the only external dependency of the root module.
16+
It's the only external dependency of the root module.
1617

1718
## Main concepts
1819

19-
| Concept | Description |
20-
| -------------- | -------------------------------------------------------------------------------------- |
21-
| **Workflow** | A repeatable process defined as a directed graph of steps |
22-
| **Step** | A node in the graph — runs an activity, joins branches, sleeps, waits, or pauses |
23-
| **Activity** | A function that performs the actual work |
24-
| **Edge** | Defines flow between steps, optionally guarded by a condition |
25-
| **Execution** | A single run of a workflow, with its own state |
26-
| **Branch** | An independent execution thread with its own copy of state |
27-
| **State** | Branch-local mutable variables that activities read and write |
28-
| **Runner** | Production entry point that composes heartbeat, timeout, resume, and completion hooks |
20+
| Concept | Description |
21+
| ------------- | --------------------------------------------------------------------------------- |
22+
| **Workflow** | A repeatable process defined as a directed graph of steps |
23+
| **Step** | A node in the graph — runs an activity, joins branches, sleeps, waits, or pauses |
24+
| **Activity** | A function that performs the actual work |
25+
| **Edge** | Defines flow between steps, optionally guarded by a condition |
26+
| **Execution** | A single run of a workflow, with its own state |
27+
| **Branch** | An independent execution thread with its own copy of state |
28+
| **State** | Branch-local mutable variables that activities read and write |
29+
| **Runner** | Convenience entry point that composes heartbeat, timeout, resume, and hooks |
2930

3031
## Quick example
3132

@@ -104,46 +105,57 @@ func main() {
104105
}
105106
```
106107

107-
The `Runner` is the recommended entry point for production code. It
108-
composes heartbeating, default timeouts, resume-from-checkpoint, and
109-
completion hooks. For one-shot scripts and tests, calling
110-
`exec.Execute(ctx)` directly is fine.
111-
112-
## Going to production
113-
114-
The library is a pure execution engine. Storage, scheduling, signal
115-
delivery, and worker leasing are the consumer's responsibility — the
116-
library defines interfaces (`Checkpointer`, `StepProgressStore`,
117-
`ActivityLogger`, `SignalStore`, `WorkflowRegistry`) and you wire
118-
your own backends. The bundled `MemoryCheckpointer` and
119-
`FileCheckpointer` are for development only.
120-
121-
See [`docs/production_checklist.md`](docs/production_checklist.md)
122-
for the full punch list, and [`docs/suspension.md`](docs/suspension.md)
123-
for the suspend / resume / replay-safety contract.
124-
125-
For a turnkey queue-backed runner and persistence backends, see
126-
[`docs/worker.md`](docs/worker.md) and
127-
[`docs/postgres.md`](docs/postgres.md). The
128-
[`experimental/worker/`](experimental/worker/),
129-
[`experimental/store/postgres/`](experimental/store/postgres/), and
130-
[`experimental/store/sqlite/`](experimental/store/sqlite/)
131-
submodules ship with this repo but live in their own Go modules so
132-
the root module stays stdlib-only. Everything under `experimental/`
133-
is functional but the API is not yet stable.
134-
135-
## Reference
136-
137-
- [`documentation/`](documentation/) — user guides covering
138-
activities, branching, checkpointing, expressions, signals/sleep/pause,
139-
state management, testing, and more.
140-
- [`llms.txt`](llms.txt) — full API reference, including the JSON
108+
For one-shot scripts and tests, calling `exec.Execute(ctx)` directly
109+
is perfectly fine. The `Runner` is the convenient wrapper when you
110+
also want heartbeating, default timeouts, resume-from-checkpoint, and
111+
completion hooks.
112+
113+
## Bring your own storage
114+
115+
The library is a pure execution engine — it doesn't ship a database,
116+
a queue, or a UI, and it never will. Instead it defines small
117+
interfaces (`Checkpointer`, `StepProgressStore`, `ActivityLogger`,
118+
`SignalStore`, `WorkflowRegistry`) and you wire up whatever backends
119+
fit your stack. The bundled `MemoryCheckpointer` and
120+
`FileCheckpointer` are great for development and tests.
121+
122+
If you'd rather not start from scratch, the `experimental/`
123+
submodules give you a head start:
124+
125+
- [`experimental/worker/`](experimental/worker/) — a queue-backed
126+
durable worker with claim loop, heartbeat, reaper, and credit
127+
reconciliation. Handlers receive a `*HandlerContext` carrying the
128+
current `Claim` plus pre-fenced stores. The
129+
[`runquery`](experimental/worker/runquery/) subpackage exposes a
130+
backend-neutral read API (`GetRun`, `ListRuns`, `CountRuns`,
131+
`DeleteRun`) so dashboards can stay storage-agnostic.
132+
- [`experimental/store/postgres/`](experimental/store/postgres/) —
133+
pgx-backed persistence that implements every store interface plus
134+
`runquery.Store`. Schema namespace is configurable via
135+
`WithSchema(...)` so it can live alongside other tables.
136+
- [`experimental/store/sqlite/`](experimental/store/sqlite/) — the
137+
same surface backed by `database/sql`. Single-writer, perfect for
138+
dev and single-process deployments.
139+
140+
These submodules have their own `go.mod`, so the root module stays
141+
stdlib-only. Their APIs are still being shaped — expect some churn.
142+
143+
## Where to look next
144+
145+
- [`documentation/`](documentation/) — friendly user guides for
146+
activities, branching, checkpointing, expressions, signals, sleep,
147+
pause, state management, testing, and more.
148+
- [`llms.txt`](llms.txt) — the full API reference, including the JSON
141149
workflow format and the script-compiler interface.
150+
- [`docs/worker.md`](docs/worker.md) and
151+
[`docs/postgres.md`](docs/postgres.md) — guides for the experimental
152+
worker and Postgres store.
153+
- [`docs/suspension.md`](docs/suspension.md) — the suspend / resume /
154+
replay-safety contract.
142155
- [`MIGRATION.md`](MIGRATION.md) — every breaking change between
143156
pre-v1 and v1, with before/after snippets.
144157
- [`examples/`](examples/) — runnable example programs covering
145-
branching, joins, retries, child workflows, and more. See
146-
[`examples/signal_wait/`](examples/signal_wait),
147-
[`examples/durable_sleep/`](examples/durable_sleep), and
148-
[`examples/pause_unpause/`](examples/pause_unpause) for the
149-
suspend/resume primitives.
158+
branching, joins, retries, child workflows, and the suspend/resume
159+
primitives ([`signal_wait`](examples/signal_wait),
160+
[`durable_sleep`](examples/durable_sleep),
161+
[`pause_unpause`](examples/pause_unpause)).

0 commit comments

Comments
 (0)