The harness, not the model, makes a coding agent good. Build one from scratch, from a bare-bones agent loop to a swarm of cloud agents.
Open-source course by Decoding AI in collaboration with Modal, Opik (by Comet) and Kitaru (by ZenML).
Try the finished agent first — 5 minutes, $0:
git clone https://github.com/decodingai-magazine/building-a-coding-agent-from-scratch-course.git cd building-a-coding-agent-from-scratch-course make install cp .env.example .env # set LLM API key uv run decodeThen type
/demo-and pick a demo — see what they do below. Full setup guide.
Type /demo- and the six demos are one keystroke away.
In LangChain's Terminal-Bench experiment, changing only the harness (with the same model) moved a coding agent from ~30th place into the top 5: the harness, not the model, is what makes a coding agent good.
agent = Agent(
build_model(settings.llm_provider), # gemini | openrouter | modal
deps_type=AgentDeps, # cwd, event sink, permission gate
output_type=[str, DeferredToolRequests], # final answer, or tools paused for approval
)
register_tools(agent) # read, edit, bash, grep, ...
async with agent.iter(prompt, message_history=history) as run:
async for node in run: # model request → tool calls → repeat
stream_events(node)That's the entire tool-calling agent. Everything else in this repo: the tools, skills, the permission layer, sandbox, steering queue, memory, compaction, durable runtime, remote execution, the subagent fan-out, the evals, is the harness. That's what you're here to build.
A fresh session powered by Qwen 3.6 35B hosted on Modal
We spent months under the hood of Claude Code (via its leaked source), OpenCode, Pi, and Aider, then distilled it into 8 articles and 4 videos where you'll build decode, your own coding agent, from scratch. One headless core hooked to two modes: an interactive TUI and a remote runtime running N copies in parallel.
Two interface modes on the left, the headless harness on the right, the evals plane underneath.
The finished agent ships with demo skills under .decode/skills/. Open the TUI, type /demo-, pick one, and watch the harness you're about to build do real work:
Implement the Skills Standard
Type /demo- and the six demos are one keystroke away.
Capable of Creating Games |
Fetching Data & Creating Dashboards |
Extracting Ontologies & Rendering Graphs |
|
And the infra that powers the agents.
Durability & Replay for AI Agents |
Remote Sandboxing |
Powered by Open Source Models |
Adding AI Evals & Observability |
- Design a coding agent harness from scratch
- Implement a headless coding agent loop
- Attach the headless harness to multiple modes: TUI and remote
- Add a runtime for durable execution, human-in-the-loop and replays when running parallel agents
- Implement guardrails and safety nets for the agent's behavior by adding a permission layer and local & remote sandboxing
- Build essential context engineering techniques: memory, compaction, skills
- Hook up an LSP server for faster feedback loops
- Implement an agents catalog: build, plan, code reviewer and exploration agents
- Spawn parallel subagents via fan-out strategies
- Add observability
- Design an eval harness for benchmarking the agent and checking for regressions
- Deploy and run swarms of agents
Plan mode, live: the agent breaks the Snake demo into a task list with the todo tool — [x] done, [~] in progress.
The code is written in Python, with the following frameworks and libraries:
- Agent Framework: Pydantic AI
- LLM Providers: Modal (open weights you serve yourself via SGLang), OpenRouter (open weights as a service), or Gemini (proprietary).
- Durable Runtime & Replays: Kitaru
- Observability & Evals: Opik
- Sandboxing: local Docker & remote Modal sandboxes
- Deploying: GCP & Modal
Otherwise, we build all the functionality from scratch, to teach you the foundations that last, not frameworks that abstract away the hard parts.
For the full experience, go through the articles and videos that cover what the code can't. The why behind every decision.
- Why we have a headless harness and two interface modes: TUI + Remote.
- What the essential components of a coding agent are, and what is optional.
- Why we plugged in 9 tools, no more, no less.
- Why we need a durable runtime and replays.
- What guardrails are actually useful.
- Why compaction fires at ~80% of the window instead of at the limit.
- Why you need benchmarks, regression tests and online evals.
| Lesson | Written Lesson | Video Lesson | Description | Running the code |
|---|---|---|---|---|
| 1 Building a Coding Agent From Scratch |
 |
🎬 Video 1 — coming soon | Designing the harness around the model, from the agent loop to a remote swarm. | 01_install_and_usage.md · 02_modal_endpoints.md |
| 2 The Bare-Bones Coding Agent Loop |
 |
🎬 Video 1 — coming soon | One agent loop, 9 tools, and a terminal you can steer. | 01_install_and_usage.md · 02_modal_endpoints.md |
| 3 From a Raw Shell to a Sandboxed Coding Agent |
 |
🎬 Video 2 — coming soon | Safely execute your agent's commands locally via Docker or remotely via Modal. | 01_install_and_usage.md · 02_modal_endpoints.md · 04_sandboxing.md |
| 4 Context Engineering for Coding Agents |
📄 Coming soon | 🎬 Video 2 — coming soon | Memory, compaction, skills, and LSP: the context window treated as a budget. | 01_install_and_usage.md · 02_modal_endpoints.md |
| 5 Agents Catalog, Subagents & Parallel Fan-out |
📄 Coming soon | 🎬 Video 2 — coming soon | One call fans out N parallel subagents, each with a budget and a report contract. | 01_install_and_usage.md · 02_modal_endpoints.md · 04_sandboxing.md |
| 6 Remote Headless Mode & Durability |
📄 Coming soon | 🎬 Video 3 — coming soon | kill -9 a headless run, resume it from checkpoints, deploy to GCP + Modal and build the same feature 5–10× in parallel. |
01_install_and_usage.md · 02_modal_endpoints.md · 03_runtime.md · 04_sandboxing.md · 06_credentials.md · 07_infra.md |
| 7 AI Evals Foundations: Benchmarks, Regression and Online |
📄 Coming soon | 🎬 Video 3 — coming soon | Benchmarks, regression probes, and online evals: does it work, still work, keep working? | 01_install_and_usage.md · 02_modal_endpoints.md · 05_evals.md |
| 8 AI Evals on Steroids via Replays |
📄 Coming soon | 🎬 Video 4 — coming soon | Replay a recorded run with the model or a checkpoint swapped, and diff the two executions. | 01_install_and_usage.md · 02_modal_endpoints.md · 03_runtime.md · 05_evals.md |
Engineers who learn by building. You finish with a working coding agent that teaches you harness engineering patterns to steal for your own agentic applications.
Best for ML/AI engineers who want to level up their craft and for software engineers and data scientists who want to transition into building agentic systems from scratch.
| Category | Requirements |
|---|---|
| Skills | - Python (Intermediate) - LLMs & agents (Beginner) |
| Hardware | Any modern machine will do. No GPU required, as we run all the LLMs in the cloud. |
| Level | Intermediate (but with a little sweat and patience, anyone can do it) |
| Time | ~4–8 hours for the whole course — 4 if you read and watch, 6–8 if you run everything. |
Running the code costs $0 if you stick to free tiers:
| Service | Cost |
|---|---|
| Gemini API (default provider — easy setup, but limited API requests) | free tier (Google AI Studio) |
| Modal (recommended provider + remote sandbox) | $30 free credits — enough to run the course |
| OpenRouter (alternative provider) | $0 on :free models (optional $10 credit raises the daily cap) |
| Opik (tracing + evals) | free tier |
| Kitaru (durable runtime) | free, runs locally offline |
| GCP — deploy the agent to run remotely (optional) | ~$16/month while it's up; new GCP accounts get $300 in credits — see 07_infra.md |
Reading-only? Everything's free!
As an open-source course, everything is self-paced, based on this repository, plus the attached lessons that walk you through the code. No paywall. No platform.
Read the lessons on the Decoding AI Magazine, watch the videos from the Decoding AI Channel, run the code on your own machine, break it, fix it, and learn from the process.
One Python package; each module maps to one part of the architecture:
.
├── docs/
│ ├── adr/ # Architecture Decision Records — the "why" of every choice
│ ├── glossary.md # one canonical name per concept
│ └── evals.md # the four-track eval suite, mapped
├── evals/ # benchmark + regression probes + demo skills
├── tests/{unit,integration}/ # mirrors src/ 1:1; milestone capstones prove each milestone
└── src/decode/
├── cli.py # Click entrypoint → launches the TUI
├── tui/ # input: prompt_toolkit · output: Rich
├── harness/ # message queue + priority gate around the loop
├── agent/ # the Pydantic-AI ReAct loop (LLM ⇄ tools)
├── agents/ # agents catalog: Build / Plan / Code-Reviewer + Explore subagent
├── tools/ # file I/O, bash, web, todo, skills dispatch, LSP, ask_user
├── permissions/ # allow/ask/deny · modes · settings.json
├── sandbox/ # bash + file tools seam: none (host) / docker / modal
├── services/lsp/ # hand-rolled stdio LSP client (ty)
├── runtime/ # Kitaru durable flow: decode run / replay / HITL
├── context/ # compaction + conversation log (JSONL)
├── memory/ # AGENTS.md / MEMORY.md loading + write-back
├── observability/ # Opik tracing
└── config/, entities/ # settings singleton · shared models
Everything lives under running_the_code/. One core guide, plus one focused guide per side quest:
| Guide | What's inside |
|---|---|
| 00_troubleshooting.md | Every known failure, and its fix |
| 01_install_and_usage.md | Start here |
| 02_modal_endpoints.md | Serving open models on Modal |
| 03_runtime.md | Runtime setup for headless mode |
| 04_sandboxing.md | Docker (local) / Modal (remote) setup for sandboxing |
| 05_evals.md | Benchmarks, regression probes, and online evals |
| 06_credentials.md | Environments & secrets, walked end-to-end |
| 07_infra.md | Deploying the remote runtime to GCP and Modal |
Special thanks to Modal, Opik (by Comet), and Kitaru (by ZenML) for sponsoring this open-source course and keeping it free!
Opik and Kitaru are open source. Consider starring their repositories: Opik on GitHub · Kitaru on GitHub.
Open a GitHub issue for course questions, setup trouble, or concept clarifications. Known gotchas are documented in the running_the_code/ guides.
Do I need a paid API key?
No. The default Gemini provider has a free tier, OpenRouter routes across :free models, and Modal gives $30 in credits — see Cost Structure.
Why Python and not TypeScript or Go? Accessibility: our audience knows Python. The course focuses on the design decisions, which transfer to any language.
Why build from scratch instead of extending Pi, DeepAgents, or an existing harness? Because adding custom logic to an existing harness is the easy part. Knowing what to add requires understanding the internals. That's the fundamentals, and it's what still makes AI engineers valuable. Build a coding agent once and you're equipped to build a custom agent for any use case.
Found a bug and know the fix? Fork, fix, run make ci (no API key needed), and open a pull request. Future readers will thank you 🤗
Paul Iusztin |
Senior AI Engineer, Educator & Founder of Decoding AI. Author of the best-selling LLM Engineer's Handbook. |
Join 40k+ engineers subscribed to the Decoding AI Magazine to learn to build coding agents from scratch.
If you found this course useful, consider starring the repository so others can find it too.
Released under Apache-2.0 — clone, fork, and build on it; keep the LICENSE and credit this repo.