Skip to content

Sketch generation cost: move to a cheaper per-token provider before scaling to ~100/day #1184

Description

@bwhitman

We want to double down on sketch generation for a new Tulip-related project, scaling from today's volume to ~100 sketches/day, without charging a subscription — so generation cost needs to be as low as possible while staying good. This issue writes up the measured cost of the current Claude Sonnet pipeline and the math on cheaper per-token alternatives (hosted open models, Haiku), including why self-hosting a fine-tuned model doesn't pencil out.

Current usage (measured from production)

Pulled from the generations table via the admin API (last 500 rows):

  • ~17 generations/day (500 gens over June 19 – July 18, 2026)
  • Model: claude-sonnet-4-6, 93% pass validation on first pass
  • Per generation (means): ~13k uncached input tokens, ~3.7k output tokens
  • Cached system prefix: ~11–12k tokens (10KB system prompt + 30KB AMY_AGENTS.md + reference-tool defs). Not logged in the DB; estimated from prompt sizes in tulip/server/amyboardworld_db_api.py. At 17/day, generations are usually >5 min apart, so most pay a fresh 5-minute-TTL cache write rather than a cheap read.

Cost per generation at Sonnet pricing ($3/M in, $15/M out, cache write 1.25×, cache read 0.1×):

Component Tokens Cost/gen
Uncached input 13k × $3/M $0.039
Output 3.7k × $15/M $0.056
Cache write (prefix) ~12k × $3.75/M $0.045
Cache reads (loop turns) ~30k × $0.30/M $0.009
Total ~$0.15/generation

That's ~$75–80/month today, and ~$450/month at 100/day.

Alternatives at 100 sketches/day

Option Per gen Per month Notes
Sonnet 4.6 (today) $0.15 ~$450 baseline
Haiku 4.5 ~$0.05 ~$150 one env-var change (AMYBOARD_GENERATE_MODEL); same caching + tool loop
Qwen3-Coder-480B (DeepInfra, $0.30/$1.00, $0.10 cached) ~$0.011 ~$35 needs an OpenAI-compat client path; good tool calling
DeepSeek V3.2 (DeepInfra, $0.26/$0.38) ~$0.009 ~$27
Qwen3-32B ($0.08/$0.28) ~$0.003 ~$10 weakest; lean on the validator

A per-token open-model host is ~13× cheaper than Sonnet for this workload.

(Open-model prices move around and aggregator pages don't all agree — re-verify on the provider's pricing page before committing.)

Why not boot our own fine-tuned model

Volume is tiny in GPU terms: 100 gens/day × ~50k tokens ≈ 5M tokens/day.

  • Dedicated GPU (A100-80GB class, ~$1.50–2/hr): ~$1,100–1,400/month at 24/7 — 3× more than just staying on Sonnet, at ~1% utilization.
  • Serverless GPU (Modal/RunPod per-second): compute maybe ~$50–100/month, but 30–60s cold starts loading a 30B model per burst — bad UX for an interactive "generate" button, and we'd own the ops.
  • Fine-tuning is probably unnecessary: the pipeline already injects domain knowledge via the system prompt + AMY_AGENTS.md + the agentic reference tools (search/read over sketches and docs). That's the mechanism a fine-tune would replicate, except the current approach stays current when docs change; a fine-tune goes stale on every AMY pin bump. If an off-the-shelf model fails, Fireworks/Together serve LoRA fine-tunes at ~base-model per-token rates, so the fallback doesn't change the economics.

Proposed plan

  1. Add an OpenAI-compatible provider path in tulip/server/amyboardworld_db_api.py next to the Anthropic one (base URL + model + key env vars), pointed at Qwen3-Coder on DeepInfra or Fireworks. The agentic reference-tool loop translates directly (tool calling supported); the existing validator + corrective retry is the quality safety net.
  2. Run the free eval: replay the 500 real prompts from the generations table through the candidate model offline and compare validator pass-rate (+ spot-check sound quality) against Sonnet's 93%. Replay cost on Qwen: ~$5.
  3. If the open model's pass rate is meaningfully worse, Haiku 4.5 is the zero-effort fallback at 3× savings.
  4. Orthogonal win if we stay on Claude at higher volume: switch the prefix cache breakpoint to ttl: "1h" — writes cost 2× but happen once an hour instead of nearly every generation (~$70/month saved at 100/day).

Sources for open-model pricing: DeepInfra pricing, DeepInfra vs Fireworks, DeepSeek pricing.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions