Skip to content

fix: avoid XADD-with-fetch atomic that aborts on older LLVM BPF backends#93

Closed
raflyhangga wants to merge 1 commit into
mainfrom
fix/xadd-fetch-abort-on-older-llvm
Closed

fix: avoid XADD-with-fetch atomic that aborts on older LLVM BPF backends#93
raflyhangga wants to merge 1 commit into
mainfrom
fix/xadd-fetch-abort-on-older-llvm

Conversation

@raflyhangga

Copy link
Copy Markdown
Contributor

Summary

  • block_rq_issue assigned req_id from __sync_fetch_and_add(gen, 1)'s return value against a shared BPF_ARRAY counter. Using the return value forces clang to lower this to a BPF XADD with the BPF_FETCH flag, which older/distro-bundled LLVM BPF backends can't legally emit — they hard-abort the whole process with LLVM ERROR: Invalid usage of the XADD return value. Since BCC embeds clang/LLVM in-process, this is an uncatchable SIGABRT, not a Python exception, so it slips right past the existing try/except around BPF init.
  • Replaced the shared atomic counter with a BPF_PERCPU_ARRAY bumped via a plain per-CPU increment, folding bpf_get_smp_processor_id() into the high bits for cross-CPU uniqueness. This removes the need for any atomic fetch-and-add entirely, sidestepping the LLVM-version dependency instead of trying to detect/gate around it. No consumer relies on strict global ordering of req_id, only per-request uniqueness/correlation, so this is behavior-neutral.
  • Documented the issue and fix in docs/COMPATIBILITY_FIXES.md (new section 5), matching the format of the other real-world compatibility fixes already recorded there.

Test plan

  • grep confirms no remaining __sync_fetch_and_add/atomic-builtin call sites in prober.c
  • Ran .github/scripts/bpf_smoke.py as root: prober.c compiles, all BPF programs load, and the in-kernel verifier passes (both default and -DENABLE_NETWORK builds), and all VFS probes attach successfully
  • Re-run on the machine that previously hit the abort to confirm it's resolved (this is LLVM-version-dependent and won't reproduce on a dev host with a newer toolchain)

🤖 Generated with Claude Code

block_rq_issue used __sync_fetch_and_add's return value against a shared
BPF_ARRAY counter to assign req_id, which lowers to a BPF XADD with the
BPF_FETCH flag. Older/distro clang BPF backends can't legally emit that
and hard-abort the whole process with "LLVM ERROR: Invalid usage of the
XADD return value" (uncatchable as a Python exception, since BCC embeds
clang/LLVM in-process). Replace the shared atomic counter with a
BPF_PERCPU_ARRAY bumped via a plain per-CPU increment, folding the CPU id
into the high bits for cross-CPU uniqueness, removing the need for any
atomic fetch-and-add entirely.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request resolves a process crash (LLVM ERROR: Invalid usage of the XADD return value) on older LLVM/BCC builds by replacing the shared atomic counter block_req_id_gen with a per-CPU array counter (BPF_PERCPU_ARRAY). The request ID is now generated by combining the SMP processor ID in the high bits with a locally incremented counter, avoiding the need for atomic fetch-and-add operations that require newer LLVM backend support. Additionally, documentation was added to docs/COMPATIBILITY_FIXES.md detailing this fix. There are no review comments, so I have no feedback to provide.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

@raflyhangga raflyhangga closed this Jul 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant