Skip to content

Commit 2b81ff4

Browse files
committed
docs(rollout_gateway): attribute vendored slime code
Add slime attribution to NOTICE (Apache-2.0) for the files adapted into the rollout gateway, and document the upstream baseline commit + re-sync workflow in AGENTS.md so future lifts can diff against the right slime revision.
1 parent 9dd7f0c commit 2b81ff4

2 files changed

Lines changed: 48 additions & 1 deletion

File tree

‎AGENTS.md‎

Lines changed: 34 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -262,13 +262,31 @@ sample-only backends like Tinker, which cannot render themselves.
262262
| `TrajectoryManager` | `trajectory.py` | Per-session message **tree**. Handles multi-turn concatenation, parallel tool-call branches, and heals re-tokenization drift between turns (CLEAN / REALIGN / FORK). Tokenizer-free and torch-free. |
263263
| `Renderer` | `render.py` | Tokenization seam. `HfTemplateRenderer` (default, HF `apply_chat_template`) or `TinkerRenderer` (needs `tinker-cookbook`, installed manually). |
264264
| `SamplingBackend` | `sampling_backends/` | The one per-engine seam: `token_ids -> token_ids + logprobs` as a `TurnRecord`. Impls: `VllmHttpBackend`, `SglangHttpBackend`, `TinkerSdkBackend`. |
265-
| Adapters | `adapters/` | Wire-protocol translation: `OpenAIAdapter` (`/v1/chat/completions`), `AnthropicAdapter` (`/v1/messages`). |
265+
| Adapters | `adapters/` | Wire-protocol translation: `OpenAIAdapter` (`/v1/chat/completions`), `AnthropicAdapter` (`/v1/messages`). An agent drives the gateway in its *native* protocol unmodified (just point `base_url` at it); both normalize to one canonical message form and share one `TrajectoryManager`. |
266266
| `RolloutGateway` | `gateway.py` | Assembles tokenizer + renderer + backend + adapters onto one aiohttp app sharing one `TrajectoryManager`. Session identity rides in the api-key / Bearer slot; `base_url` is a fixed gateway address (no per-session URLs). |
267267

268268
**Session model.** A session id (in the Bearer slot) keys one trajectory tree.
269269
`gateway.create_session(sid)` → agent turns are captured → `gateway.finish_session(sid)`
270270
drains the tree into `list[TraceRecord]`.
271271

272+
**Multi-API & sub-agents.**
273+
- *Multi-API* (works today): OpenAI- and Anthropic-protocol turns for the same session id
274+
fold into the same trajectory tree, so one training run can capture agents that speak
275+
different wire protocols.
276+
- *Dynamic sub-agents as in-session forks* (works today): when a harness spawns a sub-agent
277+
(e.g. Claude Code's Task tool) whose LLM calls reuse the **same** session id, its distinct
278+
system prompt doesn't match the parent's branch, so `TrajectoryManager` **forks** a new leaf
279+
automatically (`_find_mount_point` matches by role + message equality). `get_trajectory`
280+
walks *all* leaves, so the parent trajectory and each sub-agent trajectory are all captured
281+
and correlated under one tree — the common self-spawning-sub-agent case needs no extra
282+
wiring. (Verifying this end-to-end against a real Claude Code harness is a to-do; the design
283+
is in place, but only synthetic/single-agent flows have been exercised so far.)
284+
- *Sub-agents that run under a **distinct** session id* (grouping pending): if the harness
285+
gives a sub-agent its own session id (its own Bearer key), it becomes a separate tree. Tying
286+
those separate trees into one episode requires stamping a shared `rollout_id` across their
287+
`TraceRecord`s — that stamping lives in the (not-yet-landed) dispatch layer, so this
288+
cross-session case is not wired in this package yet.
289+
272290
**Dependencies.** The gateway is trainer-side and lives behind extras — the base install
273291
(agent-side `AgentCoreRLApp` / `RolloutClient`) stays lean:
274292
- `pip install agentcore-rl-toolkit[gateway]` → `aiohttp` + `transformers`.
@@ -288,6 +306,21 @@ backend (the per-backend rollout function + `TraceRecord → native sample` conv
288306
the ACR dispatch/reward-join glue) is not yet on the main branch — a prototype dispatcher
289307
is parked on the `wip/online-rl-dispatch` branch until a backend consumes it end-to-end.
290308

309+
**Vendored from slime (upstream baselines).** Several files are adapted from
310+
[slime](https://github.com/THUDM/slime) (Apache-2.0; see `NOTICE`). To check what changed
311+
upstream before re-syncing, diff the source file against the baseline commit below:
312+
313+
| This repo | slime source | Baseline commit |
314+
|---|---|---|
315+
| `rollout_gateway/trajectory.py` | `slime/agent/trajectory.py` | `90c212b5` |
316+
| `rollout_gateway/adapters/{common,openai,anthropic}.py` | `slime/agent/adapters/` | `90c212b5` |
317+
| `rollout_gateway/parsing.py` | `slime/agent/parsing.py` | `90c212b5` |
318+
319+
Re-sync workflow: `git -C <slime> diff 90c212b5..HEAD -- slime/agent/<file>` shows upstream
320+
changes since the lift. Our copies are intentionally modified (torch-free; `Sample` →
321+
`TraceRecord`; injected backend/renderer seams; sglang parser hook removed), so treat the diff
322+
as a review aid, not an automatic merge. Bump the baseline commit here when you re-sync.
323+
291324
### Migration Guide (basic_app → rl_app)
292325

293326
See `examples/strands_math_agent` for a complete example adapting from `basic_app.py` to `rl_app.py`.

‎NOTICE‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1 +1,15 @@
11
Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
2+
3+
------------------------------------------------------------------------------
4+
5+
This product includes software developed as part of the slime project
6+
(https://github.com/THUDM/slime), licensed under the Apache License, Version 2.0.
7+
8+
The rollout gateway (src/agentcore_rl_toolkit/rollout_gateway/) adapts code from
9+
slime, notably:
10+
- trajectory.py — TrajectoryManager, TurnRecord, MessageNode, and the
11+
CLEAN/REALIGN/FORK drift classification, adapted from
12+
slime/agent/trajectory.py (made torch-free).
13+
- adapters/ — OpenAIAdapter, AnthropicAdapter, and the shared BaseAdapter
14+
pipeline, adapted from slime/agent/adapters/.
15+
- parsing.py — output-parsing helpers, adapted from slime/agent/parsing.py.

0 commit comments

Comments
 (0)