Skip to content

Conversation

@luke-moehlenbrock
Copy link
Contributor

@luke-moehlenbrock luke-moehlenbrock commented Dec 2, 2025

Adds support for routingClassifierTraces and also splits out the guardrails to pre and post guardrails as separate spans.


Note

Adds routingClassifierTrace, separates guardrails into preGuardrailTrace/postGuardrailTrace, and enhances metadata and input/output extraction across nodes and spans.

  • Instrumentation (core):
    • AttributeExtractor: merge invocation/observation metadata for agent-collaborator I/O; include routingClassifierTrace event; improve agent-collaborator I/O attribute construction.
  • Span assembly:
    • ResponseAccumulator:
      • Prioritize node-level chunks; special handling for agent-collaborator nodes when extracting parent input/output.
      • Parse routing-classifier outputs from rawResponse.content; fall back sensibly.
      • Name guardrail node spans as preGuardrailTrace/postGuardrailTrace; set Guardrails span kind/status (ERROR when intervened).
      • Extract and merge metadata in precedence order; use metadata times to set span start/end.
    • TraceCollector: guardrail-specific unique trace IDs; add chunks to nodes (for metadata propagation); ensure new nodes (incl. guardrail) store chunks/spans correctly.
  • Tests:
    • Add cassette tests/.../cassettes/test_routing_classifier_trace.yaml and new test_routing_classifier_trace.
    • Update existing tests to assert orchestration metadata, pre/post guardrail traces, span counts/names, and metadata shapes.

Written by Cursor Bugbot for commit 70340c2. This will update automatically on new commits. Configure here.

@luke-moehlenbrock luke-moehlenbrock requested a review from a team as a code owner December 2, 2025 22:25
@dosubot dosubot bot added the size:XL This PR changes 500-999 lines, ignoring generated files. label Dec 2, 2025
assert orchestration_metadata is not None
assert isinstance(orchestration_metadata, str)
orchestration_metadata_dict = json.loads(orchestration_metadata)
assert orchestration_metadata_dict["clientRequestId"] == "edfe2c05-729a-4768-b930-602aece156b0"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm wondering if there is a better alternatives to the hard coded expected values used in these test assertions.

In the future, when we need to update/refresh the vcr recording files, these assertions would also be manually updated again, right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I completely agree, it's a huge pain to get the correct hardcoded values here, it might be better just to check if there is a non-null value in these fields. We're not modifying these values after we get them from Bedrock so just checking that they've been added to the span should be good and we don't need to check the exact value.

@dosubot dosubot bot added the lgtm This PR has been approved by a maintainer label Dec 4, 2025
@luke-moehlenbrock luke-moehlenbrock merged commit e878e1c into Arize-ai:main Dec 4, 2025
26 checks passed
@github-actions github-actions bot mentioned this pull request Dec 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

lgtm This PR has been approved by a maintainer size:XL This PR changes 500-999 lines, ignoring generated files.

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants