feat: add DeerFlow 2.x observability

[sipercai]

Description

This PR adds observability support for ByteDance DeerFlow 2.x as a thin adapter over the existing LoongSuite LangChain and LangGraph instrumentations.

The adapter marks only DeerFlow-created LangGraph graphs with two internal scalar semantics: the framework name and the direct model-decision node name. LangGraph propagates those values through a copy of RunnableConfig.metadata, and the LangChain tracer uses them to create one AGENT and exactly one STEP per direct model decision. Existing LangGraph prebuilt ReAct detection and DeepAgents behavior remain unchanged.

The DeerFlow package adds Gateway and embedded-client ENTRY lifecycle handling, lead/subagent identity, session and user propagation, correlation attributes, streaming teardown, timeout/error/interruption handling, cached-alias restoration, and concurrency isolation. Direct SDK graph invocation starts at AGENT and does not invent an application ENTRY.

The public package supports the official DeerFlow source distribution at >=2,<3 and intentionally does not depend on the unrelated PyPI placeholder. The README documents local foreground/daemon, Docker development/production, and embedded SDK integration. Local command wrapping and persistent site bootstrap do not require changes to DeerFlow Python source.

This PR does not change loongsuite-distro/src/loongsuite/distro/bootstrap_registry; release registration and version refreshes remain part of the release workflow.

Fixes # (N/A)

Type of change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

How Has This Been Tested?

• tox -e precommit
• tox -c tox-loongsuite.ini -e py312-test-loongsuite-instrumentation-deerflow.stable -- -q (52 passed)
• tox -c tox-loongsuite.ini -e py312-test-loongsuite-instrumentation-deerflow.current -- -q (52 passed)
• tox -c tox-loongsuite.ini -e py313-test-loongsuite-instrumentation-deerflow.current -- -q (52 passed)
• tox -c tox-loongsuite.ini -e lint-loongsuite-instrumentation-deerflow
• tox -c tox-loongsuite.ini -e py312-test-loongsuite-instrumentation-langchain-latest -- -q (173 passed)
• tox -c tox-loongsuite.ini -e py312-test-loongsuite-instrumentation-langgraph-latest -- -q (20 passed)
• tox -c tox-loongsuite.ini -e py312-test-loongsuite-instrumentation-deepagents-latest -- -q (11 passed)
• tox -e py312-test-util-genai -- -q (271 passed, 1 skipped because optional OSS credentials were not configured)
• PYTHONPATH=.github/workflows/generate_workflows_lib/src python .github/workflows/generate_workflows.py
• PYTHONPATH=.github/workflows/generate_workflows_lib/src python .github/workflows/generate_workflows_loongsuite.py
• Official-source auto-instrumentation smoke: loongsuite-instrument produced one AGENT -> STEP -> CHAIN -> LLM tree without duplicate core spans
• Official-source site-bootstrap smoke: plain Python produced one AGENT -> STEP -> LLM tree without an application import change
• Official DeerFlow source matrix: v2.0.0 tag, pinned current revision, and the latest upstream main

Does This PR Require a Core Repo Change?

• Yes. - Link to PR:
• No.

Checklist:

See contributing.md for styleguide, changelog guidelines, and more.

• Followed the style guidelines of this project
• Changelogs have been updated
• Unit tests have been added
• Documentation has been updated

Validation Evidence

Scope and compatibility

• Supported framework: official DeerFlow >=2,<3, Python 3.12+.
• The instrumentation has an empty static target-library dependency list and performs a runtime import/version guard.
• bootstrap_registry has no diff from main; publishing metadata is deferred to the release process.
• DeepAgents source has no diff from main.
• Plain, unmarked LangGraph graphs are not reclassified as agents. Existing LangGraph prebuilt ReAct graphs continue to use their established agent decision node.

Lifecycle and telemetry

• Gateway and embedded-client calls create at most one application ENTRY; direct graph calls begin at AGENT.
• Each model decision creates exactly one STEP; two-round tool loops report tool_calls followed by stop.
• Lead agents, subagents, concurrent threads, cross-thread generator consumption, early close, cancellation, timeout, model/tool errors, and recursion-limit failures are covered.
• LangSmith and Langfuse callbacks are preserved through copy-on-write config handling; enabling multiple LoongSuite instrumentors does not duplicate AGENT or STEP spans.
• Default mode does not capture prompt, tool-argument, or response content.

Weaver and CI

• The sanitized official-source trace passes the core tree, parent, span-kind, message, finish-reason, token, TTFT, and error checks.
• The remaining Weaver registry mismatch is gen_ai.framework, an existing LoongSuite extension attribute not yet declared by the external registry; DeerFlow-specific lifecycle validation is otherwise complete.
• The current remote PR head is green. Updating this branch will trigger GitHub checks again; the PR remains draft until the new head is green and maintainers accept or resolve the external registry gap.

[ralf0131]

• [Info] patch.py:_track_graph — Strong-ref fallback list for unhashable graphs could grow without bounds in theory, but in practice graphs are created once per application. Acceptable for current usage.

Code Quality Highlights

• Optional DeerFlow imports are correctly wrapped in try/except (telemetry must not hard-depend on application internals)
• _LEGACY_GRAPH_ATTRS cleanup ensures graphs aren't misclassified as ReAct/deepagents when marked as DeerFlow - Entry depth via ContextVar prevents nested span creation - CI workflow JSON additions are mechanical (adding deerflow to lint/test matrices) ### Test Coverage - New test files under loongsuite-instrumentation-deerflow/tests/ with conftest fixtures - test_response_id.py for the shared resolve_response_id helper (from #247's util-genai addition)

Note: Reviewed the core patch.py (802 lines) and extended_handler.py changes in detail. CI config, package metadata, and README changes reviewed at a high level.

---

Automated review by github-manager-bot

[ralf0131]

Summary

Adds a new loongsuite-instrumentation-deerflow package (802-line patch.py) for DeerFlow 2.x observability, plus a small context-flag addition to extended_handler.py and updates to langchain/langgraph instrumentations for DeerFlow graph detection. The new instrumentation follows established patterns: ContextVar-based entry depth tracking, WeakKeyDictionary graph markers with strong-ref fallback for unhashable graphs, proper baggage/context propagation, and custom exception types for run states (error/timeout/interrupted).

Findings

No blocking issues.

• [Info] extended_handler.py:131 — _ENTRY_ACTIVE_CONTEXT_KEY + is_entry_context_active() is a clean, minimal addition. Other instrumentations can now check if they're inside an ENTRY span without reimplementing detection logic.
• [Info] patch.py:_track_graph — Strong-ref fallback list for unhashable graphs could grow without bounds in theory, but in practice graphs are created once per application. Acceptable for current usage.

Code Quality Highlights

• Optional DeerFlow imports are correctly wrapped in try/except (telemetry must not hard-depend on application internals)
• _LEGACY_GRAPH_ATTRS cleanup ensures graphs aren't misclassified as ReAct/deepAgents when marked as DeerFlow
• Entry depth via ContextVar prevents nested span creation
• CI workflow JSON additions are mechanical (adding deerflow to lint/test matrices)

Test Coverage

• New test files under loongsuite-instrumentation-deerflow/tests/ with conftest fixtures
• test_response_id.py for the shared resolve_response_id helper (from fix(hermes): preserve provider response IDs #247's util-genai addition)

Note: Reviewed the core patch.py (802 lines) and extended_handler.py changes in detail. CI config, package metadata, and README changes reviewed at a high level.

---

Automated review by github-manager-bot

[ralf0131]

⚠️ Merge conflict detected

This PR has conflicts with the main branch and cannot be merged. Please rebase or merge main into your branch and resolve the conflicts:

git fetch origin
git checkout feat/deerflow-observability
git rebase origin/main
# resolve conflicts, then:
git push --force-with-lease

This is a one-time reminder. Feel free to @mention me for a re-review after conflicts are resolved.

---

Automated notification by github-manager-bot

[ralf0131]

Summary

New DeerFlow 2.x instrumentation adapter — a thin layer over existing LangChain/LangGraph instrumentations. Code structure looks solid.

Review

• Architecture ✅ — Clean BaseInstrumentor implementation with proper dependency management (LangChain + LangGraph instrumentors)
• Version gating ✅ — Correct >=2,<3 range check on deerflow-harness distribution
• Error handling ✅ — Proper cleanup on failure (reverse-order uninstrument)
• Semantic conventions ✅ — Well-defined constants for agent framework, session, and trace attributes
• Tests ✅ — Comprehensive VCR-based test coverage

⚠️ Action Required

This PR has merge conflicts with main. Please rebase and resolve conflicts before this can be merged:

git fetch origin
git checkout <your-branch>
git rebase origin/main
# resolve conflicts, then:
git push --force-with-lease

---

Automated review by github-manager-bot

[ralf0131]

Summary

Adds OpenTelemetry instrumentation for ByteDance DeerFlow 2.x. New package loongsuite-instrumentation-deerflow follows the established LoongSuite instrumentation pattern with proper version gating (2.x ≤ v < 3.x), reentrancy tracking via ContextVar, and wrapt-based function patching.

Findings

• [Info] The PR has merge conflicts with main that need to be resolved before merge.
• [Info] Core patch logic (802 lines) is well-structured. The _ENTRY_DEPTH ContextVar correctly handles reentrancy, and WeakKeyDictionary for graph markers avoids memory leaks.
• [Info] Error handling is comprehensive: DeerFlowRunError, DeerFlowRunTimeout, DeerFlowRunInterrupted provide clear error semantics.
• [Info] Consider splitting the CI workflow file additions into a separate commit for cleaner history.

Code looks good. Please resolve merge conflicts when convenient.

---

Automated review by github-manager-bot

[ralf0131]

Summary

DeerFlow 2.x observability adapter built on top of existing LangChain/LangGraph instrumentations. Clean adapter pattern with proper ENTRY/AGENT/STEP span hierarchy, Gateway and embedded-client lifecycle handling, and comprehensive test coverage (52 tests across Python 3.12-3.13).

Findings

• [Info] patch.py is 802 lines — the core instrumentation logic is substantial. Consider splitting Gateway vs embedded-client patches into separate modules if complexity grows.
• [Info] Good use of WeakKeyDictionary for graph marker tracking to avoid memory leaks, and ContextVar for entry depth to ensure thread safety.

Suggestions

• The version validation (>=2, <3) and fail-open import handling are well-implemented. The documentation clearly explains the supported deployment modes (foreground/daemon/Docker/embedded SDK).

---

Automated review by github-manager-bot