Skip to content

feat: add OpenAI Agents SDK support to OpenInference mapper - #366

Draft
liramon2 wants to merge 4 commits into
strands-agents:mainfrom
liramon2:openai-openinference
Draft

feat: add OpenAI Agents SDK support to OpenInference mapper#366
liramon2 wants to merge 4 commits into
strands-agents:mainfrom
liramon2:openai-openinference

Conversation

@liramon2

@liramon2 liramon2 commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

Description

Add OpenAI Agents SDK support to OpenInference mapper. This handles both Cloudwatch ADOT and in-memory exporter formats. No inheritance or composition is used because that is a one-way door decision that calls for a larger refactor.

This also fixes parent span ids in OpenInferenceSessionMapper so that they point to converted spans. This preserves the agent-tool scopes when converting from OpenAI Agent traces (or other OpenInference traces) to evaluator inputs.

Related Issues

#364

Documentation PR

Type of Change

New feature

Testing

How have you tested the change? Verify that the changes do not break functionality or introduce new warnings.

  • I ran hatch run prepare

Checklist

  • I have read the CONTRIBUTING document
  • I have reviewed and understand every line of code in this PR, including any generated by AI tools, and I can explain why it works
  • My change is focused and reasonably small; I have split unrelated work into separate PRs
  • I have added any necessary tests that prove my fix is effective or my feature works
  • I have updated the documentation accordingly
  • I have added an appropriate example to the documentation to outline the feature, or no new docs are needed
  • My changes generate no new warnings
  • Any dependent changes have been merged and published

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@github-actions github-actions Bot added enhancement New feature or request area-tracing Trace/session ingestion: providers, session mappers, extractors, telemetry/OTEL labels Aug 12, 2026
@liramon2
liramon2 deployed to auto-approve August 12, 2026 21:35 — with GitHub Actions Active
@liramon2

Copy link
Copy Markdown
Contributor Author

@strandly-the-agent Review this PR. Consolidate findings into a single comment.

@strandly-the-agent

This comment was marked as resolved.

@poshinchen

Copy link
Copy Markdown
Contributor

@strandly-the-agent Here are the instrumentation:

  1. "https://github.com/traceloop/openllmetry/tree/main/packages/opentelemetry-instrumentation-openai-agents",
  2. "https://github.com/traceloop/openllmetry/tree/main/packages/opentelemetry-instrumentation-openai",
  3. "https://github.com/Arize-ai/openinference/tree/main/python/instrumentation/openinference-instrumentation-openai-agents",
  4. "https://github.com/Arize-ai/openinference/tree/main/python/instrumentation/openinference-instrumentation-openai"

Here are the current findings:

For OpenAI agents SDK + traceloop instrumentation:

  • It mostly follows GenAI conventions
  • The instrumentation does not populate agent spans' available_tools . Tools only show up as descendant ToolExecution spans, so available_tools will be handled by backfilling from descendant tool spans
  • The traces have an empty root span that serves as a wrapper. Its operation is invoke_agent instead of the genai convention of invoke_workflow . These should be stripped to avoid having an empty agent span.
  • Agents that perform handoffs represent orchestrator and sub-agent spans as siblings. Need to use agent_handoff spans to correct parent ids if having valid parent ids is relevant

For OpenAI agents SDK + OpenInference instrumentation:

  • It mostly follows openinference conventions
  • input.value and output.value are not agent span attributes. They are instead on descendant LLM spans. So they are handled with a walk down to those spans and looking into llm.input_messages.N.message.content.0.message_content.text and llm.output_messages.N.message.contents.0.message_content.text
  • Available tools from _trace_tools_map are scoped to the trace while multiple agents can exist in the trace. This makes judges compare orchestrator tool calls against all available tools in the trace, causing false positives (bug seen last week). This can be handled by copying LLM tool schemas llm.tools.N.tool.json_schema onto AGENT spans and extracting them into available_tools . Granted, this can be deferred since the other frameworks using openinference also don't handle multi-agent traces correctly.

NOTE: In the current mappers' implementation strategy, I don't think we should use composition, and inheritance. And the mappers are grouped by instrumentation frameworks.

I need you to:

  1. verify the above claims,
  2. tell me how to implement the classes, should they be an additional class or just have the logics in the current mapper(s).
  3. I don't think we need inheritance for now. Tell me the suggestions

@strandly-the-agent

Copy link
Copy Markdown

Verified all seven claims against the instrumentation sources (openinference-instrumentation-openai-agents 2.0.0, opentelemetry-instrumentation-openai-agents 0.62.3 from Traceloop) and the captured fixtures in both PRs (#366 at bc6422d, #365 at ebc5c5a). Six hold; three need precision that changes the remedy.

# Claim Verdict
T1 Traceloop mostly follows GenAI conventions ✅ (deviations noted below)
T2 Traceloop doesn't populate available_tools → backfill from descendant tool spans ⚠️ true for the fixtures, but current v0.62.3 emits gen_ai.tool.definitions — prefer it; backfill has a semantic trap
T3 Empty root wrapper with invoke_agent → strip it ✅ (one precision: semconv has no invoke_workflow, and the wrapper's attrs change across versions)
T4 Handoff agents are siblings → reparent via agent_handoff spans ✅ (v0.62.3 adds a simpler signal: gen_ai.agent.handoff_parent on the sub-agent span)
O1 OpenInference mostly follows its conventions
O2 input/output live on descendant LLM spans → walk + read message paths ⚠️ mechanism right; the key paths as written are off, and index selection is the actual hard part
O3 _trace_tools_map is trace-scoped → multi-agent false positives; copy llm.tools.* onto AGENT spans; deferrable ✅ — but the copy alone doesn't close the bug you saw; the orchestrator has to survive too

Design answer (Q2/Q3): agree — no inheritance, and no new classes. openai_agents is a producer inside two convention families, not a framework of its own: keep its logic inside the two family mappers (OpenInferenceSessionMapper, GenericGenAISessionMapper) as scope-gated hooks in a fixed three-phase shape — normalize → shared convert → post-link. Applied consistently, that means folding #365's subclass back into GenericGenAISessionMapper. Details and a sketch below.

Traceloop verification (T1–T4)

T1 — GenAI conventions ✅. Fixture chat spans carry gen_ai.operation.name (chat/execute_tool/invoke_agent), the new semconv message format (gen_ai.input.messages/gen_ai.output.messages as role/parts JSON), gen_ai.usage.*, gen_ai.agent.name/id/description. Deviations to plan for: per-turn wrapper spans with gen_ai.operation.name: "unknown" (8–12 attrs, no content); the custom agent_handoff operation (not in semconv); deprecated gen_ai.system alongside gen_ai.provider.name.

T2 — ⚠️ version-dependent, and the backfill has a semantic trap.

  • In both feat(mappers): add OpenAI Agents OTel session mapper #365 fixtures the claim is exactly right: the only tool-ish attributes anywhere are gen_ai.tool.call.{name,arguments,result,type} on execute_tool spans — zero declared-tool schemas.
  • But the current release emits the real declared list: v0.62.3's _end_generation_span sets gen_ai.tool.definitions (full name/description/parameters JSON, via _extract_tool_definitions) on generation/response spans when content tracing is on (_hooks.py:1021-1028; runs for GenerationSpanData/ResponseSpanData, :800-803). The feat(mappers): add OpenAI Agents OTel session mapper #365 fixtures simply predate it.
  • So the remedy should be: prefer gen_ai.tool.definitions when present, fall back to backfill. Two reasons the backfill alone is worth improving: (i) descendant ToolExecutionSpans are tools used, not tools available — a tool-selection judge sees a shrunken candidate set with the un-chosen options missing, which is the mirror image of the false-positive bug you saw on the OpenInference side (candidate set too big there, too small here); (ii) feat(mappers): add OpenAI Agents OTel session mapper #365's backfill builds ToolConfig(name=...) only — no description/parameters for the judge to reason over, while gen_ai.tool.definitions carries both.

T3 — ✅ confirmed in both fixtures. Root Agent workflow span with gen_ai.operation.name: "invoke_agent" and no agent name / no messages (live: 5 attrs, all provider/server boilerplate). Stripping is right, and #365's rule (no gen_ai.agent.name and no gen_ai.input.messages → skip) is the robust one — better than keying on the operation name, because the wrapper's attributes are version-dependent: in v0.62.3 the root gets traceloop.span.kind: workflow and no gen_ai.operation.name at all (on_trace_start, _hooks.py:676-688). One nit on the claim's wording: GenAI semconv defines no invoke_workflow operation (chat, execute_tool, invoke_agent, create_agent, embeddings, …) — workflow-ness is Traceloop's own traceloop.span.kind, so there's no "correct" operation the wrapper should have had; it just needs stripping.

T4 — ✅ confirmed. In the ADOT fixture, invoke_agent coordinator and invoke_agent math_specialist are siblings (children of the same turn wrapper under the root), with the agent_handoff span (gen_ai.handoff.from_agent: coordinator, gen_ai.handoff.to_agent: math_specialist) sitting under the coordinator's turn — the name-based reparenting in #365's _apply_handoff_reparenting is the right join, and the ebc5c5a tiebreaker doc covers the duplicate-name case. Two additions:

  • The agents-as-tools pattern needs no correction — in the live fixture, execute_tool ask_math_specialist → nested invoke_agent math_specialist nests properly already. Sibling-flattening is handoff-specific.
  • v0.62.3 also stamps gen_ai.agent.handoff_parent directly on the handed-off agent's own invoke_agent span (utils.py:14, applied in _start_agent_span via _reverse_handoffs_dict). When present it's a simpler, collision-free signal than the handoff-span join — worth preferring, with the span-join as fallback for older captures like the fixture.
OpenInference verification (O1–O3)

O2 — mechanism ✅, paths and indices need precision. Verified against the v2.0.0 instrumentor source and both #366 fixtures:

  • Input: plain-text messages carry the scalar llm.input_messages.N.message.content — that's the only input shape in both shipped fixtures. The contents.0.message_content.text shape appears for structured/multimodal input (_get_attributes_from_message_param emits one or the other, not both). The path as written in your comment (...message.content.0.message_content.text) mixes the two — a mapper reading only that would drop every plain-text trace. Read scalar first, contents.0...text as fallback (that's also exactly review finding 1c: the current PR code reads only the scalar and drops multimodal spans).
  • Output: contents.0.message_content.text on v2.0.0, scalar .message.content on v1.6.1 — both fallbacks needed (the PR does this part right).
  • The hard part is N, not the path: the Responses-API path numbers input from 1 (_get_attributes_from_input, msg_idx=1), the chat-completions path (GenerationSpanData — LiteLLM/Azure/compatible endpoints) from 0, and reasoning items consume output indices. So: highest-index message with role == "user" for input, highest-index non-reasoning text for output — hardcoded indices break multi-turn, reasoning models, and the chat-completions path (review finding 1, with repros).

O3 — ✅ verified, with one important limit. _trace_tools_map[trace_id] is populated from every LLM span in the trace during conversion, so in a multi-agent trace each agent's available_tools is the trace-wide union — your false-positive mechanism is exactly what my LLM-context pass reproduced (a judge asked to justify handoff to math_specialist({}) against a candidate set that doesn't contain it). Copying llm.tools.N.tool.json_schema onto AGENT spans fixes the scoping, but as shipped it doesn't close the bug you saw, for two reasons:

  • The orchestrator's AGENT span is currently dropped (its final LLM turn is tool-calls-only, so output.value never gets injected and the _is_agent_invocation_span gate rejects it). Its handoff tool span is then misattributed to the surviving specialist by the orphan fallback (types/trace.py:202-206) and judged against the specialist's tools — same species of false positive, one hop over. The fix needs the orchestrator to survive (or orphaned tool spans to stay unowned), not just the tool copy. That's review blocker 2 / Question 1 — it's the decision this hinges on.
  • The copy takes schemas from the earliest LLM span only, so per-turn tool filtering (FunctionTool.is_enabled, MCP without cache) loses entries. Deriving indices from keys and merging turns is cheap.
  • The "deferrable because other openinference producers don't handle multi-agent either" part is accurate: langchain dedups to one agent span (is_langchain gate), and smolagents/claude multi-agent traces get the same trace-wide union today.
Design recommendation (Q2/Q3): no new classes, no inheritance — three named phases inside the two family mappers

The rule that matches the repo you already have: one mapper per emitting-convention family (OpenInferenceSessionMapper already serves langchain/smolagents/claude via scope-gated tweaks; GenericGenAISessionMapper serves GenAI-convention traces), routed by detect_otel_mapper on scope. openai_agents is a producer inside two families — OpenInference scope in #366, GenAI scope in #365 — so it doesn't earn a class in either. A per-producer class would duplicate each family's conversion machinery (span-kind detection, message parsing, ADOT handling) for an 80-line delta.

Why I'd agree about inheritance specifically: #365's subclass works by overriding _convert_trace / _convert_agent_invocation_span — it's coupled to GenericGenAISessionMapper calling those exact template methods in that exact order. A refactor of the base silently changes the subclass's behaviour with no test failing in the subclass's own file (fragile-base-class). Scope-gated hooks keep the coupling visible in one place. Composition (a strategy object per producer) would also work but buys nothing over a plain method-dispatch table at this size — it's structure without payoff.

The shape I'd standardize (both mappers already approximate it):

  1. Normalize (per-producer, before shared conversion): rewrite raw span dicts into the family's canonical shape. _normalize_smolagents_span and _normalize_openai_agents_trace are this today. Registered in a dispatch table rather than a growing if-chain:
    # producer scope -> (granularity, hook)
    _PRODUCER_NORMALIZERS = {
        SCOPE_OPENINFERENCE_SMOLAGENTS: ("span", _normalize_smolagents_span),
        SCOPE_OPENINFERENCE_OPENAI_AGENTS: ("trace", _normalize_openai_agents_trace),
    }
  2. Convert (shared, producer-blind): one code path per family. Producer conditionals should not appear here — if one is needed, the data should have been normalized instead.
  3. Post-link (per-producer where needed, after conversion): structural fixes on converted spans — feat(mappers): add OpenAI Agents OTel session mapper #365's handoff reparenting and tools backfill live here; bridge_parent_gaps is the family-agnostic version and stays shared in utils.

Three guardrails, all from bugs already found in these two PRs: each hook is exception-isolated (one producer's quirk must degrade to a warned-and-skipped trace, not kill the session — the #366 normalizer currently runs outside every try/except); each hook checks scope on the spans it touches (the #366 walk doesn't); and each hook gets producer fixtures that pin it (mutation testing showed the bridge_parent_gaps call can be deleted with all 385 tests green).

Applied to the two PRs:

  • feat: add OpenAI Agents SDK support to OpenInference mapper #366: keep everything in OpenInferenceSessionMapper — no new class. The changes it needs are the review fixes plus, optionally, the dispatch table above.
  • feat(mappers): add OpenAI Agents OTel session mapper #365: the consistent no-inheritance answer is to fold OpenAIAgentsOtelSessionMapper into GenericGenAISessionMapper: wrapper-strip becomes a normalize/skip rule, handoff-reparent + tools-backfill become a post-link step, both gated on SCOPE_OPENAI_AGENTS, and detect_otel_mapper:140-141 routes to the generic mapper. It's ~118 lines whose three behaviours are already discrete methods, so folding is cheapest now, before it grows. Keeping the subclass is workable, but then the codebase carries two different strategies for the same problem — which is the divergence that prompted your question.

One thing I'd not do: don't introduce a producer-level abstraction (base class or protocol for "producers") until a third family needs it. Two call sites is a pattern; codifying it early is how the mapper layer gets rigid.

Verification by strandly-the-agent: claims checked against instrumentor sources (openinference 2.0.0, traceloop 0.62.3) and both PRs' fixtures; OpenInference items re-verified with the repros from my review above. As ever — my analysis to check, your call to make.

@liramon2

Copy link
Copy Markdown
Contributor Author

@strandly-the-agent Review this PR. Consolidate findings into a single comment. For your questions:

  • The coordinator should exist, but the workflow wrapper should not
  • They can land in any order. The parallel fix was a coincidence
  • No live integration tests yet

@strandly-the-agent

strandly-the-agent commented Aug 15, 2026

Copy link
Copy Markdown

Re-reviewed at 1c9587f (fix: address strandly + refactor: consolidate tool schema normalization, base 1f752dc). Follow-up shape: I re-ran every round-1 repro cold against the new head, mutation-tested the key gap, traced judge payloads through TraceExtractor, and ran a battery of fresh adversarial probes against the new helper code (the independent subagent pass hit its harness time limit, but its probe scripts survived and I executed all of them myself — findings folded in below).

Verdict: no blockers remain — both round-1 🔴s are fixed and verified. Three 🟡 should-fixes before un-drafting:

  1. Pin bridge_parent_gaps — deleting the call leaves the whole suite green while multiply_numbers silently flips ownership to the coordinator (verified test supplied).
  2. Image-first multimodal input still drops the agent span_get_last_message_text reads only part 0; a [image, text] user message has its text at part 1.
  3. A unicode-digit attribute key crashes the whole sessionseg.isdigit() admits '²'/'①' but int(seg) raises, and the ValueError escapes map_to_session. One-word guard; and it means the un-wrapped normalizer call has a live trigger again.
Round-1 finding Status at 1c9587f
🔴 1 — hard-coded attribute paths (5 manifestations) fixed — multi-turn ✓, reasoning ✓, multimodal (text part 0) ✓, chat-completions indexing ✓, tool-call-only turn ✓ ([delegated] fallback)
🔴 2 — handoff collapse + misattribution fixed — coordinator + specialist both survive; handoff owned by the coordinator; per-agent tools; judge payloads verified end-to-end
🟡 3 — normalizer crash escaping map_to_session ✅ both round-1 triggers fixed — but the probes found a new one (🟡 3 below), so the wrap suggestion is back on the table
🟡 4 — test gaps mostly fixed — 10 new tests pin the helpers/counts/crash cases; bridge_parent_gaps remains unpinned
⚪ hunk-G non-dict-schema regression fixed properly — unwrap moved into a scope-gated per-span normalizer with an isinstance check

Suites: mapper 389 passed (was 385) · extractors/types/detectors 235 passed.

Your three answers, closed out: coordinator exists / wrapper doesn't — implemented exactly that way and verified in both fixtures. Ordering with #365 — noted, no dependency. No integ tests yet — matches the #340#353 precedent; a tracking issue would close the loop but isn't blocking.

Fix verification — round-1 repros re-run at 1c9587f, verbatim
multi-turn:        user_prompt='Now divide that by 5.'  response='555 divided by 5 is 111.'  (was: turn-1's question)
reasoning @ idx 0: response='REAL ANSWER'                                                    (was: the scratch-pad)
multimodal input:  span survives, prompt='MULTIMODAL Q'                                      (was: dropped)
chat-completions:  input numbered from 0 → prompt='CC QUESTION'                              (was: wrong index)
tool-call-only:    response='[delegated] transfer_to_x({})'                                  (was: dropped)
het. start_time:   no crash; healthy LangChain trace in the same batch survives              (was: TypeError killed the session)
attributes=None:   no crash; degrades to a per-span warning                                  (was: AttributeError killed the session)
non-dict schema:   tools=[] and the span survives                                            (was: whole span dropped)
OpenAI envelope:   ('calc', description, parameters) intact via the new per-span normalizer

Residual from manifestation (e): an empty-string final answer still drops the span (if agent_response: + the gate). The reachable case — tool-call-only — is fixed; the empty-string one is rare, ⚪.

The new unit tests (test_get_last_message_text_uses_current_turn_user_message, test_get_last_message_text_skips_reasoning_item, test_normalization_survives_heterogeneous_start_times, test_normalization_survives_none_attributes_span) pin the helper behaviours — these were among the 10 surviving mutants last round, so that's the right set to have added.

Coordinator survival — verified end-to-end, including what the judge sees

ADOT fixture: 3 raw AGENT spans → 2 AgentInvocationSpans (wrapper dropped, per your intent):

coordinator:      response='[delegated] transfer_to_math_specialist(); transfer_to_research_specialist()'
                  tools=['transfer_to_math_specialist', 'transfer_to_research_specialist']
math_specialist:  response='The product of 42×7 is 294. As for the weather…'
                  tools=['add_numbers', 'divide_numbers', 'multiply_numbers']
TOOL 'handoff to math_specialist' → agent_span_id = coordinator      ✓
TOOL 'multiply_numbers'           → agent_span_id = math_specialist  ✓

TOOL_LEVEL judge payload — this closes the false-positive poshinchen reported (handoff judged against the wrong candidate set):

target: handoff to math_specialist  available_tools=['transfer_to_math_specialist', 'transfer_to_research_specialist']
target: multiply_numbers            available_tools=['add_numbers', 'divide_numbers', 'multiply_numbers']

The recaptured live fixture (agents-as-tools, 19 spans) also maps cleanly: wrapper dropped, coordinator (with ask_* tools) + specialist both survive, and the specialist's prompt is the actual tool-call argument it received.

One inherent oddity, flagged so nobody rediscovers it as a bug: both agents carry the same user_prompt (the SDK threads the full conversation into the handed-off agent), so the specialist's TRACE_LEVEL history shows the user question twice. Faithful to what the specialist actually received — ⚪.

🟡 1 — pin bridge_parent_gaps (verified test supplied)

With the call at openinference_session_mapper.py:423 replaced by pass, the mapper file's 127 tests all pass — and multiply_numbers.agent_span_id flips from math_specialist to coordinator via the orphan fallback (types/trace.py). So the suite stays green while tool attribution goes wrong in exactly the way this PR fixes. test_handoff_tool_owned_by_coordinator doesn't catch it: the handoff lands on the coordinator through the root-fallback whether or not the bridge runs.

This test passes with the call and fails without it (verified both ways):

def test_multiply_numbers_owned_by_math_specialist(self, openai_agents_adot_session):
    """Pins bridge_parent_gaps: without parent-gap bridging, the orphan fallback
    hands multiply_numbers to the coordinator instead of the specialist that ran it."""
    all_spans = [s for t in openai_agents_adot_session.traces for s in t.spans]
    specialist = next(
        s for s in all_spans
        if isinstance(s, AgentInvocationSpan) and "[delegated]" not in (s.agent_response or "")
    )
    multiply = next(
        s for s in all_spans if isinstance(s, ToolExecutionSpan) and s.tool_call.name == "multiply_numbers"
    )
    assert multiply.agent_span_id == specialist.span_info.span_id
    assert multiply.span_info.parent_span_id == specialist.span_info.span_id
🟡 2 & 🟡 3 — new findings from the adversarial probes (both reproduced end-to-end)

🟡 2 — image-first multimodal user message drops the agent span. _get_last_message_text reads contents.0.message_content.text only. The instrumentor emits multimodal parts in order, so a user message shaped [image, text] has contents.0.type = "image" and its text at contents.1 — the helper returns None, input.value is never injected, and the AGENT span is dropped (reproduced: text-first [text, image] works; image-first yields (no traces)). Same reachability as round-1's 1c — the documented way to send images — which this fix only half-covered. Fix: scan the message's contents.N parts for the first message_content.type == "text" instead of hardcoding part 0. (Related ⚪, same loop: a multi-part text output returns only part 0, truncating the response.)

🟡 3 — _get_message_indices crashes the whole session on a unicode-digit key. seg.isdigit() is true for '²' and '①', but int(seg) raises ValueError: invalid literal for int() — and both call paths (the per-span tools normalizer and the trace normalizer) sit outside any try/except, so one malformed key kills the entire map_to_session call, other producers' traces included (reproduced end-to-end on both paths). No real instrumentor emits such keys — this is corrupt/adversarial data — but the blast radius is total and the fix is one word: if seg.isascii() and seg.isdigit(). It also retires my appendix note that the un-wrapped normalizer call had “no known trigger”: it has one now, so the round-1 try/except logger.warning wrap around the normalizer calls is worth adding after all.

Appendix — non-blocking (9)
  1. Stale provenance comment: the live-fixture header (test file ~:2517-2519) still says “single-agent … v1.6.1”, but the recaptured fixture is multi-agent (19 spans, agents-as-tools) and its outputs use the contents.N paths (v2.0.0 shape). Worth updating — the provenance comment is what the next maintainer trusts.
  2. ⚪ The descendant walk still has no visited set — a duplicate span_id closing a cycle still hangs (contrived input; one-line fix if you want it).
  3. ⚪ The tools-copy loop keeps the contiguous-index assumption (probe-confirmed: a gap at llm.tools.1 copies only tool_0 onto the AGENT span, which then shadows the fuller trace-wide fallback) and copies from the earliest LLM span only — real instrumentor output is contiguous, so ⚪.
  4. ⚪ Same contiguity in _get_last_tool_calls: tool_calls at indices {0, 2} render only index 0; a nameless call at 0 hides a named one at 1. Not real instrumentor output.
  5. [delegated] synthesis embeds raw argument JSON unbounded — a 200KB tool argument becomes a 200KB agent_response in the judge prompt. A length cap would be kind to judge budgets.
  6. ⚪ An empty-string last message now silently returns the previous message's text (input and output alike) rather than dropping — arguably worse than a drop when it happens, but the trigger (empty final content alongside earlier text in the same span) is contrived.
  7. ⚪ A message carrying reasoning at contents.0 and its real text at contents.1 is skipped entirely — v2.0.0 emits reasoning as its own message, so not real output today; the part-scan fix for 🟡 2 would cover this too.
  8. ⚪ A dict-valued (non-string) wrapped tool schema now loses the tool (json.loads(dict)TypeError → skip, and the extractor no longer unwraps) — OTEL attribute values are strings, so only hand-built input hits it.
  9. ⚪ The walk still doesn't scope-check spans in mixed-scope trace groups — needs a non-openai AGENT ancestor of openai spans to matter; contrived.

Follow-up review by strandly-the-agent: self-run re-verification of all prior findings per follow-up protocol (every repro above executed at 1c9587f), plus adversarial probes of the new helper code — the independent subagent pass timed out at its harness limit, so I recovered its probe scripts and executed the full battery myself; a full mutation sweep of the new helpers didn't complete, though the new unit tests pin the behaviours that survived mutation last round. Solid work on the fixes — the remaining 🟡s are a test and two small guards, not redesign.

@liramon2

Copy link
Copy Markdown
Contributor Author

@strandly-the-agent Review this PR. Consolidate findings into a single comment.

@strandly-the-agent

Copy link
Copy Markdown

Re-reviewed at c304ade (fixes-only delta, +79/−38; self-run follow-up, every check below executed by me at this head).

All three 🟡s from my last review are fixed and verified — nothing left that I'd block or should-fix. From my side this is ready to come out of draft.

Finding Status at c304ade
🟡 1 — pin bridge_parent_gaps ✅ test adopted (test_multiply_numbers_owned_by_math_specialist) — re-ran the mutation: deleting the call now fails 1 test (was 0)
🟡 2 — image-first multimodal drop _join_content_text scans all parts — [image, text] now yields the prompt; bonus: multi-part text outputs are joined ('Part A. Part B.'), retiring an appendix item
🟡 3 — unicode-digit key crash isascii() and isdigit() guard and both normalizer paths wrapped in try/except logger.warning — the round-1 wrap suggestion, now in; a poisoned key degrades to a warning and every trace survives

Also cleaned up from the appendix, verified: cycle guard (seen set — the duplicate-span_id cycle now terminates), gap-tolerant tools copy ({0, 2} → both tools), dict-valued wrapped schemas handled, stale fixture header comments replaced.

Regression spot-checks all green: multi-turn + reasoning still resolve correctly, ADOT fixture still yields 2 agents with per-agent tools, mapper suite 390 passed · downstream 235 passed.

Still-open ⚪ (all contrived inputs, previously disclosed, fine to leave): empty-string final answer drops the span; an empty-string last message falls back to the previous message's text; a same-message [reasoning, text] shape is skipped (is_reasoning checks part 0 only — not real v2.0.0 output); [delegated] embeds unbounded argument JSON; _get_last_tool_calls index contiguity; no scope check in mixed-scope trace groups.

Follow-up review by strandly-the-agent — fixes-only delta, so per protocol this was a self-run verification pass (mutation re-run, repro battery re-run, fixtures end-to-end), not a full pipeline. Nice, thorough round of fixes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area-tracing Trace/session ingestion: providers, session mappers, extractors, telemetry/OTEL enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants