We want to double down on sketch generation for a new Tulip-related project, scaling from today's volume to ~100 sketches/day, without charging a subscription — so generation cost needs to be as low as possible while staying good. This issue writes up the measured cost of the current Claude Sonnet pipeline and the math on cheaper per-token alternatives (hosted open models, Haiku), including why self-hosting a fine-tuned model doesn't pencil out.
Current usage (measured from production)
Pulled from the generations table via the admin API (last 500 rows):
- ~17 generations/day (500 gens over June 19 – July 18, 2026)
- Model:
claude-sonnet-4-6, 93% pass validation on first pass
- Per generation (means): ~13k uncached input tokens, ~3.7k output tokens
- Cached system prefix: ~11–12k tokens (10KB system prompt + 30KB
AMY_AGENTS.md + reference-tool defs). Not logged in the DB; estimated from prompt sizes in tulip/server/amyboardworld_db_api.py. At 17/day, generations are usually >5 min apart, so most pay a fresh 5-minute-TTL cache write rather than a cheap read.
Cost per generation at Sonnet pricing ($3/M in, $15/M out, cache write 1.25×, cache read 0.1×):
| Component |
Tokens |
Cost/gen |
| Uncached input |
13k × $3/M |
$0.039 |
| Output |
3.7k × $15/M |
$0.056 |
| Cache write (prefix) |
~12k × $3.75/M |
$0.045 |
| Cache reads (loop turns) |
~30k × $0.30/M |
$0.009 |
| Total |
|
~$0.15/generation |
That's ~$75–80/month today, and ~$450/month at 100/day.
Alternatives at 100 sketches/day
| Option |
Per gen |
Per month |
Notes |
| Sonnet 4.6 (today) |
$0.15 |
~$450 |
baseline |
| Haiku 4.5 |
~$0.05 |
~$150 |
one env-var change (AMYBOARD_GENERATE_MODEL); same caching + tool loop |
| Qwen3-Coder-480B (DeepInfra, $0.30/$1.00, $0.10 cached) |
~$0.011 |
~$35 |
needs an OpenAI-compat client path; good tool calling |
| DeepSeek V3.2 (DeepInfra, $0.26/$0.38) |
~$0.009 |
~$27 |
|
| Qwen3-32B ($0.08/$0.28) |
~$0.003 |
~$10 |
weakest; lean on the validator |
A per-token open-model host is ~13× cheaper than Sonnet for this workload.
(Open-model prices move around and aggregator pages don't all agree — re-verify on the provider's pricing page before committing.)
Why not boot our own fine-tuned model
Volume is tiny in GPU terms: 100 gens/day × ~50k tokens ≈ 5M tokens/day.
- Dedicated GPU (A100-80GB class, ~$1.50–2/hr): ~$1,100–1,400/month at 24/7 — 3× more than just staying on Sonnet, at ~1% utilization.
- Serverless GPU (Modal/RunPod per-second): compute maybe ~$50–100/month, but 30–60s cold starts loading a 30B model per burst — bad UX for an interactive "generate" button, and we'd own the ops.
- Fine-tuning is probably unnecessary: the pipeline already injects domain knowledge via the system prompt +
AMY_AGENTS.md + the agentic reference tools (search/read over sketches and docs). That's the mechanism a fine-tune would replicate, except the current approach stays current when docs change; a fine-tune goes stale on every AMY pin bump. If an off-the-shelf model fails, Fireworks/Together serve LoRA fine-tunes at ~base-model per-token rates, so the fallback doesn't change the economics.
Proposed plan
- Add an OpenAI-compatible provider path in
tulip/server/amyboardworld_db_api.py next to the Anthropic one (base URL + model + key env vars), pointed at Qwen3-Coder on DeepInfra or Fireworks. The agentic reference-tool loop translates directly (tool calling supported); the existing validator + corrective retry is the quality safety net.
- Run the free eval: replay the 500 real prompts from the
generations table through the candidate model offline and compare validator pass-rate (+ spot-check sound quality) against Sonnet's 93%. Replay cost on Qwen: ~$5.
- If the open model's pass rate is meaningfully worse, Haiku 4.5 is the zero-effort fallback at 3× savings.
- Orthogonal win if we stay on Claude at higher volume: switch the prefix cache breakpoint to
ttl: "1h" — writes cost 2× but happen once an hour instead of nearly every generation (~$70/month saved at 100/day).
Sources for open-model pricing: DeepInfra pricing, DeepInfra vs Fireworks, DeepSeek pricing.
We want to double down on sketch generation for a new Tulip-related project, scaling from today's volume to ~100 sketches/day, without charging a subscription — so generation cost needs to be as low as possible while staying good. This issue writes up the measured cost of the current Claude Sonnet pipeline and the math on cheaper per-token alternatives (hosted open models, Haiku), including why self-hosting a fine-tuned model doesn't pencil out.
Current usage (measured from production)
Pulled from the
generationstable via the admin API (last 500 rows):claude-sonnet-4-6, 93% pass validation on first passAMY_AGENTS.md+ reference-tool defs). Not logged in the DB; estimated from prompt sizes intulip/server/amyboardworld_db_api.py. At 17/day, generations are usually >5 min apart, so most pay a fresh 5-minute-TTL cache write rather than a cheap read.Cost per generation at Sonnet pricing ($3/M in, $15/M out, cache write 1.25×, cache read 0.1×):
That's ~$75–80/month today, and ~$450/month at 100/day.
Alternatives at 100 sketches/day
AMYBOARD_GENERATE_MODEL); same caching + tool loopA per-token open-model host is ~13× cheaper than Sonnet for this workload.
(Open-model prices move around and aggregator pages don't all agree — re-verify on the provider's pricing page before committing.)
Why not boot our own fine-tuned model
Volume is tiny in GPU terms: 100 gens/day × ~50k tokens ≈ 5M tokens/day.
AMY_AGENTS.md+ the agentic reference tools (search/read over sketches and docs). That's the mechanism a fine-tune would replicate, except the current approach stays current when docs change; a fine-tune goes stale on every AMY pin bump. If an off-the-shelf model fails, Fireworks/Together serve LoRA fine-tunes at ~base-model per-token rates, so the fallback doesn't change the economics.Proposed plan
tulip/server/amyboardworld_db_api.pynext to the Anthropic one (base URL + model + key env vars), pointed at Qwen3-Coder on DeepInfra or Fireworks. The agentic reference-tool loop translates directly (tool calling supported); the existing validator + corrective retry is the quality safety net.generationstable through the candidate model offline and compare validator pass-rate (+ spot-check sound quality) against Sonnet's 93%. Replay cost on Qwen: ~$5.ttl: "1h"— writes cost 2× but happen once an hour instead of nearly every generation (~$70/month saved at 100/day).Sources for open-model pricing: DeepInfra pricing, DeepInfra vs Fireworks, DeepSeek pricing.