Skip to content

Latest commit

 

History

History
134 lines (105 loc) · 11 KB

File metadata and controls

134 lines (105 loc) · 11 KB
title Builder embedding it in their own product — ADOPT-WITH-SCARS
description Reference documentation for Builder embedding it in their own product — ADOPT-WITH-SCARS, preserving the page's implementation details, evidence, and operating context.

Builder embedding it in their own product — ADOPT-WITH-SCARS

Building a product on top, so they inherit every constraint permanently and cannot route around one later. They will spend three engineer-weeks evaluating, and they weight portability far above everyone else, because for them a narrow model matrix is not an inconvenience -- it is a ceiling on their own product.

  • Next best option overall — build directly on an agent SDK plus a serving engine, and own the glue
  • Tolerance — 120 engineer-hours of unfamiliarity before this buyer walks away. Every cell's load is measured against it.
  • Switch bar+0.50: the w_eff one axis must clear before switching is rational.
  • VerdictADOPT-WITH-SCARS. run-integrity carries it; you eat steady-state-ops to get it
facet weight w_eff verdict β load vs
Run integrity 0.10 +1.00 CRUISE +1.00 +0.25 an agent loop that trusts its own completion report
Session longevity 0.10 +0.74 CRUISE +0.76 +0.25 no context management -- run until the window fills
Token economy 0.10 +0.72 CRUISE +0.75 +0.25 tuned warm per-agent KV cache
Observability 0.15 +0.68 CRUISE +0.67 +0.12 OpenTelemetry GenAI tracing (LangSmith, Langfuse, OTel)
Injection control 0.15 +0.44 DRIFT +0.00 -0.42 formal-isolation defenses (CaMeL, MELON)
Raw speed 0.10 -0.51 DRAG -0.25 +0.25 a raw SGLang endpoint, unmediated
Portability 0.20 -0.71 DRAG -0.43 +0.25 a raw SGLang endpoint, unmediated
Steady-state ops 0.10 -1.06 REGRESSIVE -0.67 +0.25 a raw SGLang endpoint, unmediated

Run integrity — +1.00 (CRUISE)

When you are not watching, does the run end in a state you can trust and resume?

  • fak 1 — MODELED, tools/loopgate verified-vs-naive-loop.report.json: governed arm cuts false-done 0.50 -> 0.00, with one-holder leases and runaway/injection caps
  • next best 0.5 — an agent loop that trusts its own completion report, the ungoverned arm of the same report
  • ceiling 1 (definitional)
  • β +1.000w +3.800
  • adoption 30 h fak vs 0 h alternative, of 120 h → load +0.25τ +0.255
  • w_net +3.545capped at CRUISE (MODELED evidence) → w_eff +0.999
  • cost basis POLICY.md 1,726 w + EXTENDING.md 2,867 w -- embedding the governance surface in their own control flow
  • Fence Authored corpus, not field measurement; capped at CRUISE by the MODELED provenance rule.

Session longevity — +0.74 (CRUISE)

How long can one session keep going before context forces you to start over?

  • fak 0.24 — OBSERVED, compaction_budget=48000 in docs/nightrun/cache-savings.jsonl against a 200,000-token window
  • next best 1 — no context management -- run until the window fills, definitional: an unmanaged client holds the whole history and then stops
  • ceiling 0 (definitional)
  • β +0.760w +0.996
  • adoption 30 h fak vs 0 h alternative, of 120 h → load +0.25τ +0.255
  • w_net +0.741capped at RELATIVISTIC (OBSERVED evidence) → w_eff +0.741
  • cost basis compaction is on by default; the cost is understanding its interaction with their own state model
  • Fence Scores the enforced resident budget, not work preserved across a fire. A builder inherits the compaction policy's behaviour permanently, so the unmeasured part is the part they should care about most.

Token economy — +0.72 (CRUISE)

For the same delivered work, how much of the input-token bill does it remove?

  • fak 0.983 — MEASURED, BENCHMARK-AUTHORITY.md headline row, headline-qwen-50x5.json: 1 unit of prefill work against naive re-prefill's 60.3, eliminating 0.983
  • next best 0.932 — tuned warm per-agent KV cache, the tuned arm of the same artifact: 4.1 units of 60.3, eliminating 0.932
  • ceiling 1 (definitional)
  • β +0.750w +0.973
  • adoption 30 h fak vs 0 h alternative, of 120 h → load +0.25τ +0.255
  • w_net +0.718
  • cost basis EXTENDING.md 2,867 w + docs/serve-config.md 3,286 w -- embedding the serving path, not just configuring it
  • Fence Measured on Qwen2.5-1.5B Q8_0, 50 turns x 5 agents, single host. A product builder inherits this ratio only for workloads shaped like that measurement.

Observability — +0.68 (CRUISE)

After the fact, from one artifact, can you reconstruct why it did what it did?

  • fak 0.8 — OBSERVED, the session journal records the consequential decision classes -- model traffic, cache reuse, compaction, tool-policy verdicts, recovery -- post-hoc with no live attach. Scored 4 of 5, not 5 of 5: all five classes are RECORDED, but not in one artifact -- the session journal, docs/nightrun/cache-savings.jsonl, and the /metrics split are separate surfaces, so a full reconstruction means joining across them. Everything is captured; single-artifact reconstruction is not.
  • next best 0.4 — OpenTelemetry GenAI tracing (LangSmith, Langfuse, OTel), OTel GenAI conventions cover the model call and partially the cache; no span for a compaction, a tool verdict, or a recovery decision. 2 of 5 classes
  • ceiling 1 (definitional)
  • β +0.667w +0.805
  • adoption 30 h fak vs 15 h alternative, of 120 h → load +0.12τ +0.126
  • w_net +0.679capped at RELATIVISTIC (OBSERVED evidence) → w_eff +0.679
  • cost basis EXTENDING.md 2,867 w + docs/fak/tutorial.md 4,252 w -- the read to embedding rather than merely running; alternative: instrumenting their own glue and shipping a tracing integration
  • Read The strongest reason for this buyer to build on fak rather than on an SDK plus an engine: the evidence surface is the part they would otherwise have to invent, and inventing it well is a product in itself.
  • Fence Capability decomposition, not a benchmark; capped at RELATIVISTIC by the OBSERVED provenance rule.

Injection control — +0.44 (DRIFT)

Can hostile text inside the data the agent reads make it act for the attacker?

  • fak 0 — MEASURED, BENCHMARK-AUTHORITY.md AgentDojo row: full-stack ASR 0/38 = 0.000 versus 29/38 = 0.763 detection-only
  • next best 0 — formal-isolation defenses (CaMeL, MELON), CaMeL ASR 0, MELON 0.0-2.4% -- already at the definitional floor
  • ceiling 0 (definitional)
  • β +0.000w +0.000 · mode parity_at_ceiling
  • adoption 30 h fak vs 80 h alternative, of 120 h → load -0.42τ -0.444
  • w_net +0.444
  • cost basis fak: POLICY.md 1,726 w plus embedding the guard in their control flow; alternative: designing their product's data flow so untrusted content cannot reach the control path -- a foundational architecture choice, not an integration
  • Read Parity on the number, and a real saving on the shape -- but a builder should weigh the other direction too: the formal-isolation approach is a property of THEIR architecture and travels with their product, whereas fak's is a property of a dependency.
  • Fence Benign and under-attack utility on the same suite is NEEDS_KEY.

Raw speed — -0.51 (DRAG)

How fast do tokens actually come out, against the hardware's physical limit?

  • fak 0.75 — MEASURED, BENCHMARK-AUTHORITY.md gateway row, compare.json: fak-gateway 1085.6 vs raw-SGLang 1451.6 tok/s at C=64
  • next best 1 — a raw SGLang endpoint, unmediated, the raw-SGLang arm of the same comparison
  • ceiling 1 (physical)
  • β -0.250w -0.255 · mode pure_tax
  • adoption 30 h fak vs 0 h alternative, of 120 h → load +0.25τ +0.255
  • w_net -0.511
  • cost basis same embedded serving path as the token-economy cell
  • Read Worst measured point; converges to ~3% at saturation. A builder inherits whichever end of that range their users actually run at, which they do not control.

Portability — -0.71 (DRAG)

Once you adopt it, what can you still swap out -- model, backend, agent, host?

  • fak 0.5714 — MEASURED, docs/HARDWARE-MATRIX.md support-maturity matrix (GENERATED by fak support-maturity-scorecard --write-doc, CI freshness-gated): 32 of 56 cells SUPPORTED across 14 model families x 4 backends -- 24 FENCED, 0 PROOF-PATH-ONLY, 0 UNDEFINED
  • next best 1 — a raw SGLang endpoint, unmediated, building directly on a serving engine leaves the model x backend question to that engine, which is the ceiling for this axis by construction
  • ceiling 1 (definitional)
  • β -0.429w -0.458 · mode pure_tax
  • adoption 30 h fak vs 0 h alternative, of 120 h → load +0.25τ +0.255
  • w_net -0.714
  • cost basis GPU.md 2,574 w + EXTENDING.md 2,867 w -- understanding which cells are SUPPORTED and what a FENCED cell does at runtime
  • Read This buyer weights portability higher than anyone else (0.20) because a narrow model matrix becomes a ceiling on THEIR product, permanently. Across the whole grid it is 32/56; the shape matters more than the fraction -- all 14 families run on CPU, but only 6 of 14 on any accelerator, so a product whose users have GPUs inherits the narrower number.
  • Fence A FENCED cell refuses honestly rather than returning a wrong answer, which is good engineering. This facet scores portability only; docs/HARDWARE-MATRIX.md scores the honesty and grades it A. Also note BENCHMARK-AUTHORITY.md:106 still reports 19/56 for this instrument -- a stale snapshot; the generated doc is authoritative.

Steady-state ops — -1.06 (REGRESSIVE)

After you have adopted it, what does it cost every month to keep it running?

  • fak 10 — MODELED, authored estimate from the measured adoption surface, 2026-08-10: 100 top-level verbs and 34 root operating documents, and a builder tracks the dependency's releases and behaviour changes on behalf of every one of their own users
  • next best 6 — a raw SGLang endpoint, unmediated, authored estimate: tracking a serving engine's releases and their own glue
  • ceiling 0 (definitional)
  • β -0.667w -0.805
  • adoption 30 h fak vs 0 h alternative, of 120 h → load +0.25τ +0.255
  • w_net -1.060capped at CRUISE (MODELED evidence) → w_eff -1.060
  • cost basis same embedded path as the other cells in this segment
  • Read Doubling the recurring bill lands on the display floor by the ceiling's construction. For a builder the burden is worse than the number suggests, because they absorb their dependency's churn on behalf of every downstream user rather than just themselves.
  • Fence Authored estimates grounded in the measured surface. The ratio is the claim.