Skip to content

[RFC]: Bounded asynchronous hidden-state fan-out to independent consumers #810

Description

@heiheiha798

Motivation.

Speculators can generate hidden states online for training, but it does not currently
provide a bounded shared data plane for multiple independent trainer processes. In a
1-producer/3-consumer topology, issuing one producer request per consumer repeats
prefill work. Driving all consumers from one synchronous cursor instead makes a
heterogeneous group advance at the pace of its slowest member.

The goal is to let each single-GPU consumer advance at its own rate while reusing a
producer result whenever consumer windows overlap. Retention must be bounded by the
active windows rather than by the length of a 100k-step or longer training run.

This is intentionally independent-process fan-out, not tensor or data parallelism:
each consumer has its own model, optimizer, cursor, and GPU.

Proposed Change.

Add an opt-in bounded asynchronous artifact path around the existing online training
flow:

  1. A filesystem artifact cache fingerprints a hidden-state request, coalesces
    concurrent misses, and publishes a packed result atomically. Existing online and
    offline paths remain unchanged unless the new CLI options are supplied.
  2. Each consumer registers an independent cursor and a lookbehind/lookahead window.
    The trainer process is the only authority that commits cursor progress; DataLoader
    workers may acquire authorized positions but cannot move the window.
  3. Artifacts remain reusable while they belong to the union of live consumer windows.
    Publications outside every window are evicted after active leases and packed-batch
    references are released. A lagging consumer may regenerate an evicted item instead
    of forcing faster consumers to wait.
  4. A bounded asynchronous scheduler prioritizes demand misses and fills lookahead in
    the background. Queue, in-flight, and producer capture-batch limits provide
    backpressure, so memory and producer work cannot grow with total training steps.
  5. Failure state is explicit and retryable. Cleanup, publication, lease, timeout, and
    recapture accounting is exposed for validation rather than silently treating a
    partial artifact as a cache hit.
  6. A benchmark harness runs fresh 1P1C and 1P3C scenarios serially, rejects distributed
    launchers or shared GPUs, validates logical-request/service-completion accounting,
    and records a common post-warmup interval with role-aware NVML telemetry.

The initial data plane uses a shared POSIX filesystem and therefore requires reliable
flock, same-filesystem atomic rename, and directory fsync behavior. It is not a
claim of GPU-direct or arbitrary-NFS support. The cache/coordinator boundary is kept
separate so another transfer backend can be considered later.

The proposed branch also includes opt-in DFlash consumer improvements needed to avoid
moving the bottleneck from the producer to avoidable optimizer and loss overhead:
fused AdamW with integrated gradient clipping, and a pinned optional Liger fused linear
cross-entropy path for memory-constrained runs. The default training behavior is
preserved.

Any Other Things.

The state-machine tests exercise 10k- and 100k-position streams and verify that retained
state is bounded by active windows. Focused affected tests currently report 181 passed
and 1 skipped; the compiled optional Liger CUDA forward/backward test passed separately.
Quality checks pass on Python 3.10, 3.12, and 3.13.

In three aligned Qwen3-8B DFlash 1P3C runs (3072-token sequences, 256 anchors, block
sizes 4/8/16, 30 warmup and 150 measured steps per consumer), aggregate steady consumer
throughput was 6.3735, 6.2856, and 6.5702 steps/s. The median per-consumer rates were
2.2643, 2.3135, and 1.6978 steps/s. Producer utilization reached 100% over the common
steady interval, showing that the consumers advance independently and that producer
capacity, rather than a synchronous consumer barrier, is the remaining limit in this
configuration.

I would like maintainer feedback on the cache/window contract and filesystem scope
before opening the implementation PR.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions