Skip to content

block: tag swap-origin (REQ_SWAP) requests in the flags column#95

Merged
1a1a11a merged 2 commits into
mainfrom
claude/io-tracer-issue-94-eo76d9
Jul 7, 2026
Merged

block: tag swap-origin (REQ_SWAP) requests in the flags column#95
1a1a11a merged 2 commits into
mainfrom
claude/io-tracer-issue-94-eo76d9

Conversation

@1a1a11a

@1a1a11a 1a1a11a commented Jul 7, 2026

Copy link
Copy Markdown
Member

Closes #94.

Swap I/O goes through the block layer with no filesystem counterpart, so on memory-constrained hosts it masquerades as filesystem block I/O — inflating apparent write amplification and polluting block cacheability curves. This tags swap-origin requests with a lowercase swap sub-flag in the block stream's existing free-form flags column (alongside sync/meta/ahead), so analysis can subtract it cleanly. No schema/column change.

How it works

The issue's sketch needed two corrections found while implementing:

  1. The bit cannot come from the block_rq_issue handler. A BCC TRACEPOINT_PROBE only receives the tracepoint's declared fields (dev, sector, nr_sector, rwbs, comm) — there is no struct request * in args on any kernel, and mainline block_rq_* tracepoints never carried cmd_flags at all (only patched vendor kernels do, per the repo's existing HAS_CMD_FLAGS sniffing). The CO-RE/BPF_CORE_READ suggestion also doesn't apply to this BCC-based tracer. Instead, a new best-effort kprobe on blk_mq_start_request() — the exported function that fires trace_block_rq_issue, 1:1 with the tracepoint on every kernel ≥ 5.0 — reads rq->cmd_flags & REQ_SWAP with the running kernel's own headers supplying the correct bit position (it moves across versions). The issue's submit_bio alternative was rejected: at submit_bio entry the sector is still partition-relative and bi_bdev is the partition, so (dev, sector) keys would never match the disk-absolute request-level events for swap partitions.

  2. REQ_SWAP tags swap-out only. Verified in mainline v5.15/v6.6/v6.12 mm/page_io.c: the kernel sets REQ_SWAP exclusively on swap writes; swap-in reads are plain REQ_OP_READ. That is the half the motivating write-amplification analysis needs (pswpout traffic), but pswpin correlation is not possible with this mechanism on any kernel. Documented as a caveat.

Data flow

  • Kprobe at request start: swap requests store a timestamped marker in a new (dev, sector)-keyed LRU map (block_swap_flags, sized like its siblings); every non-swap request scrubs the key, mirroring how block_start_times/block_insert_times are refreshed by each new claimant, so a marker orphaned by a missed completion cannot mis-tag a later request reusing the key.
  • block_rq_complete honors a marker only when consistent with the matched issue context: marker_ts <= ictx->ts, no older than 1 s, and a write op (rwbs[0] == 'W' — a swap-tagged read is definitionally bogus). The marker is deleted on every completion path.
  • The emitted is_swap byte is appended to the flags column in userspace via a unit-testable FlagMapper.append_swap_flag.

Degradation

  • Kernels without REQ_SWAP (< 4.19): the handler is compiled out via #ifdef; KernelProbeTracker probes for it with load_func before attaching, so old kernels don't pay a per-request kprobe trap for a no-op.
  • Missing/unprobeable symbol: skipped best-effort — block events simply carry no swap tag, nothing else affected.
  • Bio-based swap devices (zram) never produce block_rq_* events, so they are invisible to the block stream with or without this change (documented).

Docs

BLOCK_IO_EVENTS.md gains a "Swap-origin tagging" section (mechanism + caveats); the flags field row and schema.py description mention swap. Also aligned stale wording in the same doc with the code: rwbs sub-flags live in the flags column (not operation), the correlation key is a struct with separate dev/sector fields, and mainline tracepoints never exposed cmd_flags.

Testing

  • New SwapFlagTests in tests/test_flag_mapper.py (kernel-free, matching the file's pattern), including a test mirroring the _print_event_block pipeline shape. Full suite: 136 tests pass (13 skipped, as before).
  • The BPF changes were adversarially reviewed (kernel-version guards, key-formula match against the tracepoint's disk_devt/blk_rq_trace_sector formulas, map lifecycle) but cannot be exercised end-to-end here: that requires a swap-active host with root, e.g. a low-RAM VM under stress-ng --vm, checking swap-tagged write rows against /proc/vmstat pswpout — as the issue itself notes.

🤖 Generated with Claude Code

https://claude.ai/code/session_0181MzEpBhHkug2Ftfg53G4L


Generated by Claude Code

Swap I/O goes through the block layer with no filesystem counterpart, so
on memory-constrained hosts it masquerades as filesystem block I/O,
inflating apparent write amplification and polluting cacheability
analysis. Tag it so analysis can subtract it cleanly (issue #94).

The rwbs string never encodes REQ_SWAP and the block_rq_* tracepoint
args carry no cmd_flags on mainline kernels, so the bit is read from
struct request by a new best-effort kprobe on blk_mq_start_request()
(the function that fires trace_block_rq_issue). Swap requests store a
timestamped marker in a (dev, sector)-keyed LRU map; every other
request scrubs the key. The completion handler honors a marker only
when it is consistent with the matched issue context (no newer than
the issue timestamp, no older than 1s, and a write op — the kernel
sets REQ_SWAP only on swap writes), then emits is_swap, which
userspace appends to the block flags column as a lowercase "swap"
sub-flag alongside sync/meta/ahead.

The BPF handler is compiled out on kernels without REQ_SWAP (< 4.19);
KernelProbeTracker probes for it with load_func before attaching so
such kernels do not pay a per-request kprobe trap for a no-op. If the
symbol or handler is unavailable, block events simply carry no swap
tag and nothing else is affected.

Known limitations (documented in BLOCK_IO_EVENTS.md): swap-in reads
are never tagged (the kernel only flags writes), and bio-based swap
devices (e.g. zram) never appear in the block stream at all.

Also aligns stale wording in BLOCK_IO_EVENTS.md with the code: rwbs
sub-flags live in the flags column (not operation), the correlation
key is a struct with separate dev/sector fields, and mainline
tracepoints never exposed cmd_flags.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0181MzEpBhHkug2Ftfg53G4L
Copilot AI review requested due to automatic review settings July 7, 2026 03:05
Validates on a real kernel that trace_blk_mq_start_request compiles in
(REQ_SWAP defined), passes the verifier, and attaches — a broken #ifdef
guard would otherwise pass BPF() load silently and only surface at
tracer startup.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0181MzEpBhHkug2Ftfg53G4L

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces swap-origin tagging (the swap flag) for block I/O events to distinguish swap-out traffic from filesystem-origin block I/O. This is implemented via a best-effort BPF kprobe on blk_mq_start_request that captures the REQ_SWAP flag from struct request at issue time and propagates it to the completion event. Feedback recommends adding a defensive NULL check for the rq pointer in the new kprobe handler to prevent potential dereference issues.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

* guaranteed no-op.
*/
#ifdef REQ_SWAP
int trace_blk_mq_start_request(struct pt_regs *ctx, struct request *rq) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

For defensive programming and consistency with other probe entry points in prober.c, it is highly recommended to explicitly check if the rq pointer is NULL before performing any member accesses or dereferences.

int trace_blk_mq_start_request(struct pt_regs *ctx, struct request *rq) {
  if (!rq)
    return 0;

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Declining this one: the kernel never calls blk_mq_start_request() with a NULL rq (the function unconditionally dereferences it), and under BCC every member access in a kprobe handler is rewritten into bpf_probe_read_kernel(), which fails safely on a bad pointer — disk would then read as NULL and the existing if (!disk) guard returns. An explicit if (!rq) would be unreachable code.


Generated by Claude Code

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds swap-origin tagging to the Linux block I/O stream by appending a lowercase swap sub-flag into the existing free-form flags column (no schema/column additions), enabling downstream analysis to separate swap-out traffic from filesystem-origin block writes.

Changes:

  • Add a best-effort kprobe on blk_mq_start_request() to capture REQ_SWAP at request start and carry it to block_rq_complete via an (dev, sector)-keyed marker map.
  • Merge the is_swap marker into the emitted block flags string in userspace (FlagMapper.append_swap_flag) and add unit tests.
  • Update schema and block-trace documentation to describe the swap flag and its caveats (swap-out only, best-effort availability, request-based devices only).

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
tests/test_flag_mapper.py Adds unit tests covering swap flag merging behavior in the block flags pipeline.
src/tracer/FlagMapper.py Adds append_swap_flag() helper to append swap into the block flags column.
src/tracer/IOTracer.py Appends swap into the block flags output based on the emitted is_swap marker.
src/tracer/prober/prober.c Emits an is_swap marker on block completions by correlating a REQ_SWAP marker captured at request start.
src/tracer/KernelProbeTracker.py Attaches the new blk_mq_start_request kprobe best-effort (and skips it when unavailable).
src/tracer/schema.py Updates the block stream flags column description to include swap.
docs/traces/BLOCK_IO_EVENTS.md Documents swap-origin tagging mechanism and caveats; aligns operation/flags column semantics.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +3111 to +3123
#if LINUX_VERSION_CODE >= KERNEL_VERSION(5, 17, 0)
struct gendisk *disk = rq->q->disk;
#else
struct gendisk *disk = rq->rq_disk;
#endif
if (!disk)
return 0;
u32 dev = ((u32)disk->major << 20) | (u32)disk->first_minor;
struct block_rq_key_t key = block_rq_key(dev, rq->__sector);

if (rq->cmd_flags & REQ_SWAP) {
u64 ts = bpf_ktime_get_ns();
block_swap_flags.update(&key, &ts);

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is already what happens under the hood: the tracer is BCC-based, and BCC's Clang rewriter converts member accesses on kprobe arguments into bpf_probe_read_kernel() calls automatically, one per hop of a multi-level chain (rq->q->disk included), compiled against the running kernel's headers. Direct-deref style is the file's existing idiom for kprobe handlers (e.g. file && file->f_inode in the VFS handlers); explicit bpf_probe_read_kernel() appears where the rewriter can't transform the access (array copies out of tracepoint args). Hand-written probe reads here would produce the same bytecode with more code. The BPF Compile CI job builds this handler as-is.


Generated by Claude Code

@1a1a11a 1a1a11a merged commit c999960 into main Jul 7, 2026
3 checks passed
@1a1a11a 1a1a11a deleted the claude/io-tracer-issue-94-eo76d9 branch July 7, 2026 13:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

block: tag swap/pagefile-origin requests in the flags column

3 participants