Skip to content

Record after-only tool observations as spans#91

Open
VACInc wants to merge 1 commit into
comet-ml:mainfrom
VACInc:opik-after-only-tool-spans
Open

Record after-only tool observations as spans#91
VACInc wants to merge 1 commit into
comet-ml:mainfrom
VACInc:opik-after-only-tool-spans

Conversation

@VACInc

@VACInc VACInc commented May 10, 2026

Copy link
Copy Markdown

Summary

  • Record an immediate Opik tool span when after_tool_call arrives with a toolCallId but no matching open span from before_tool_call.
  • Deduplicate repeated after-only tool observations by runId + toolCallId so native replay or duplicate hook delivery does not double-record a tool.
  • Keep the existing paired before_tool_call / after_tool_call path unchanged for normal OpenClaw-owned tools.
  • Companion OpenClaw PR: Expose native tool completions to plugin hooks openclaw/openclaw#80372

cc @vincentkoc

Root Cause

The plugin assumed every after_tool_call had a previously-created tool span from before_tool_call. That was true for the old OpenClaw-owned tool path, but native tool observations can arrive as after-only completion telemetry when the runtime owns tool execution. When no matching span existed, the plugin returned early and dropped the tool result, so completed native tool calls could disappear from Opik even when OpenClaw emitted a valid after_tool_call observation.

Real Behavior Proof

Behavior fixed: Opik records after-only tool observations as completed tool spans instead of dropping them.

Before:

Code path: src/service/hooks/tool.ts searched active.toolSpans by toolCallId or tool name and returned immediately when no matching span was found.
Existing regression test: the previous no-ops when no matching tool span test asserted that an after-only event created no tool span.

After:

npm test -- src/service.test.ts

Test Files  1 passed (1)
Tests  78 passed (78)

Observed result:

  • The updated after-only test now creates a child tool span under the active LLM span from an after_tool_call event carrying toolCallId, input params, result, run id, and duration metadata.
  • The duplicate test sends the same after-only event twice and proves only one span is created, updated, and ended.

Verification

  • npm test -- src/service.test.ts
  • npm run lint
  • npm run build
  • git diff --check

What was not tested

  • Live upload to an Opik backend was not tested; the unit tests verify the plugin span creation/update/end behavior through mocked Opik traces and spans.
  • A packaged npm release was not produced; this PR only changes source and tests.

@VACInc VACInc changed the title Fix after-only Codex native tool spans Record after-only tool observations as spans May 10, 2026
@VACInc

VACInc commented May 10, 2026

Copy link
Copy Markdown
Author

Update: companion OpenClaw core PR is now openclaw/openclaw#80372 with the generic native tool completion observability naming. I also updated the local installed opik-openclaw runtime from this branch before rerunning oc-update; the updater applied #80372 and completed health/build/daemon checks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant