go-sieve - SIEVE cache eviction for Go

A Go implementation of the SIEVE cache eviction algorithm (NSDI'24, Zhang et al.), engineered from the ground up for highly concurrent, read-heavy workloads. Generic over key and value types.

The read path (Get()) is fully lock-free — a single atomic load on the key→index map plus a single atomic bit update on a shared visited bitfield. No mutex, no pointer chasing, no per-entry allocations, zero GC traffic on hits. Under concurrent read load this is 13–55x faster than hashicorp/golang-lru per trace on real-world cache traces (warm cache, 10 cores), and the gap grows with core count: SIEVE's aggregate throughput improves as cores are added while mutex-bound LRU/ARC throughput degrades below its own single-core line. Under a cold-cache mixed read/write replay, SIEVE is 2–6x faster. The write path uses a single short-held mutex and a pre-allocated node pool, so Add()/Probe() also avoid per-operation heap allocation in steady state.

If you need a cache that many goroutines hit simultaneously on the read path — an HTTP response cache, a DNS resolver cache, an authz decision cache, a hot-path object lookup — this implementation is built for that shape. On a single goroutine the two are comparable (SIEVE replays most traces faster sequentially, LRU wins a few); under any concurrency, Sieve wins decisively.

SIEVE uses a FIFO queue with a roving "hand" pointer: cache hits set a visited bit (lazy promotion), and eviction scans from the hand clearing visited bits until it finds an unvisited node (quick demotion). It matches or exceeds LRU/ARC hit ratios with far less bookkeeping — validated here on ~300M requests from the MSR Cambridge and Meta Storage trace repositories.

Key Design Elements

Array-backed indexed list. Marc Brooker observed that SIEVE's mid-list removal prevents a simple circular buffer. Rather than Tobin Baker's "swap tail into hole" workaround, this implementation uses a doubly-linked list with int32 indices into a pre-allocated backing array. This preserves SIEVE's exact eviction semantics while eliminating all interior pointers — the GC sees a flat []node with no pointers to trace (for non-pointer K, V types).

xsync.MapOf for concurrent access. The key→index map uses puzpuzpuz/xsync.MapOf which stores int32 values inline in cache-line-padded buckets — no traced pointers per entry. Get() is fully lock-free; only Add()/Probe() (on miss) and Delete() take the global mutex.

Columnar slot state (slotState). Each node's lock and visited counter are hoisted out of the node struct into a separate contiguous []uint64 — one word per slot, laid out as a column alongside the []node array. This columnar split has three effects:

1. the eviction hand scans IsVisited by walking a dense []uint64 sequentially — 8 slots per cache line, hardware-prefetch friendly — without pulling in key/val data it doesn't need;
2. Get()'s LockAndMark writes only to the slotState word for that slot, so it never dirties the cache line holding the node's key/val — no false sharing with concurrent readers of adjacent nodes
3. []uint64 contains no pointers, so the GC never traces it, unlike node fields that may hold pointer-typed K/V.

Within each word, bit 63 is a spinlock and the low bits are a saturating visited counter (1 bit at k=1, ⌈log₂(k+1)⌉ bits for higher k).

Pre-allocated node pool. All nodes are allocated once at cache creation in a contiguous array. A bump allocator + intrusive freelist (reusing node.next) provides O(1) alloc/free with zero heap allocations during steady-state operation.

TOCTOU-safe concurrent writes. Add() and Probe() use a double-check pattern: fast-path Load() outside the lock, re-check under mu.Lock() before inserting. This prevents duplicate nodes from concurrent writers racing on the same key.

Benchmark Results

Benchmarked against hashicorp/golang-lru v2.0.7 (LRU and ARC) on an Apple M4 (10 cores: 4 performance + 6 efficiency):

• macOS (Darwin 25.5.0, arm64)
• go1.26.3
• GOMAXPROCS=10

Benchmarks live in bench/ as a separate module to avoid polluting go.mod. Every table below is generated by bench/cmd/mktables from the raw results files — including derived claims, which are computed rather than hand-counted.

Commands used (no name filter — every benchmark in every package runs):

cd bench && make bench    # synthetic comparison, count=3
cd bench && make trace    # trace replay + miss ratio + GC, count=1
cd bench && make sweep SWEEP_PROCS="1 2 4 8 10"   # core-scaling sweep

Full raw results: bench-results.md.

Reading the parallel ns/op numbers

The single-digit ns/op numbers in the parallel tables below are real but need context: they are aggregate throughput.

Go's b.RunParallel distributes b.N total operations across GOMAXPROCS goroutines and reports ns/op = wall_clock / b.N. When 10 goroutines complete Get()s at 5.8 ns/op aggregate, the system produces one completed Get every ~5.8 ns; the per-core latency is ~58 ns (5.8 × 10 cores), consistent with two L1/L2-hot atomic operations plus the surrounding map lookup.

LRU/ARC report ~140–390 ns/op under the same conditions — not because a single Get is that much slower, but because every Get takes a mutex. All goroutines serialize through one lock, so adding cores makes aggregate throughput worse (see the core-scaling table below: LRU goes from 31 ns/op at 1 core to 182 ns/op at 10 — lock handoff costs grow with contention). On a single goroutine, SIEVE and LRU are in the same league — the gap is a concurrency-scaling story.

After warmup, the working data structures (map buckets, node array, visited bitfield) are resident in CPU cache — L1/L2 for small traces, L3 for larger ones. Real workloads with lower temporal locality will see higher per-core latency, but the relative advantage over mutex-bound LRU/ARC holds whenever there is any parallelism at all.

Parallel Micro-Benchmarks (count=3, medians)

Benchmark	Sieve	LRU	ARC
Get_Parallel	5.76 ns/op, 0 B	144.2 ns/op, 0 B	144.9 ns/op, 0 B
Add_Parallel	89.8 ns/op, 8 B	212.0 ns/op, 40 B	273.3 ns/op, 74 B
Probe_Parallel	91.2 ns/op, 8 B	—	—
Delete_Parallel	81.2 ns/op, 0 B	69.5 ns/op, 0 B	88.3 ns/op, 0 B
Mixed_Parallel (60/30/10)	71.0 ns/op, 2 B	162.2 ns/op, 12 B	211.8 ns/op, 23 B
Zipf_Get_Parallel (s=1.01)	24.7 ns/op	127.7 ns/op	133.1 ns/op
Zipf_Get_Parallel (s=1.50)	94.2 ns/op	100.4 ns/op	112.6 ns/op
Memory @ 1M fill	78.4 MB	112.4 MB	112.3 MB
GCImpact (1M entries)	4.37 ms/iter, 18.1 µs pause	9.46 ms/iter, 22.4 µs	9.93 ms/iter, 23.5 µs

Probe_Parallel is SIEVE-only — LRU's PeekOrAdd and ContainsOrAdd skip recency promotion and are not semantic equivalents. Delete_Parallel is the one micro where SIEVE loses: LRU's single-lock linked-list unlink edges out SIEVE's slot-state clear (69.5 vs 81.2 ns/op). At extreme Zipf skew (s=1.50) SIEVE's advantage narrows to ~6%: most requests hit a handful of keys, so readers contend on the same per-slot word.

Core scaling (Get_Parallel, median ns/op)

GOMAXPROCS	Sieve	LRU	ARC
1	14.9	31.4	33.8
2	8.9	74.3	77.0
4	5.5	121.1	146.0
8	8.0	178.7	182.0
10	7.5	182.1	193.4

SIEVE's aggregate throughput improves with cores (the 8–10 core numbers include the M4's efficiency cores); LRU/ARC throughput degrades with every added core because all readers fight over one mutex — at 10 cores LRU is ~6x slower per completed Get than at 1 core.

Trace Replay Results

This implementation is benchmarked against real-world cache traces from the libCacheSim trace repository — 13 MSR Cambridge enterprise block I/O traces + 5 Meta Storage (Tectonic) block traces totalling ~300M requests. Each trace was replayed with a cache sized at 10% of unique keys, comparing SIEVE (k=1, k=2, k=3) against hashicorp/golang-lru (LRU and ARC). Parallel benchmarks walk the trace as a ring with per-goroutine staggered start offsets (see bench/README.md).

Parallel Get throughput (warm cache, 10 goroutines, ns/op, zero allocs)

SIEVE's lock-free Get() is 13–55x faster than the better of LRU/ARC on every trace under concurrent read load:

Trace	SIEVE k=1	SIEVE k=3	LRU	ARC
msr_2007/msr_web_2	2.24	2.05	105.7	150.4
msr_2007/msr_proj_4	2.84	3.87	157.4	220.8
msr_2007/msr_prxy_0	3.28	3.15	106.3	133.3
msr_2007/msr_prn_1	3.51	3.37	135.9	185.8
meta_storage/block_traces_2	4.01	4.31	129.4	211.4
meta_storage/block_traces_1	4.03	4.34	132.5	197.9
msr_2007/msr_usr_2	4.62	4.79	209.4	268.5
meta_storage/block_traces_4	4.88	4.87	159.2	250.5
meta_storage/block_traces_3	4.95	4.56	147.2	224.8
meta_storage/block_traces_5	5.00	4.92	152.2	222.5
msr_2007/msr_src1_0	5.65	5.60	157.4	384.9
msr_2007/msr_usr_1	6.39	7.73	252.6	308.0
msr_2007/msr_src1_1	7.04	7.20	165.9	276.9
msr_2007/msr_proj_1	7.31	7.21	255.1	307.3
msr_2007/msr_proj_0	8.15	8.55	110.4	136.9
msr_2007/msr_hm_0	9.12	9.38	117.6	141.8
msr_2007/msr_proj_2	10.09	7.96	260.1	299.6
msr_2007/msr_prn_0	12.20	12.34	173.2	154.2

The k=3 saturating counter adds little to no overhead on the read path. This benchmark pre-warms the cache with a full trace replay, then hammers Get() only — the ideal scenario for a read-heavy cache in steady state.

Parallel Replay throughput (cold cache, mixed read+write, 10 goroutines, ns/op)

This is Probe() for SIEVE and Get+Add for LRU/ARC, hammered in parallel with no warmup. It measures the steady-state workload a real cache faces: reads and writes interleaved, evictions happening live. The previous table's numbers reflect the best-case read ceiling; this table reflects what throughput you actually get when your cache is doing work.

Trace	SIEVE k=1	SIEVE k=3	LRU	ARC
msr_2007/msr_prxy_0	37.4	36.6	229.3	251.1
msr_2007/msr_hm_0	58.7	61.0	228.9	297.9
msr_2007/msr_prn_0	62.2	67.6	251.6	322.3
msr_2007/msr_proj_0	63.0	58.6	236.2	312.2
msr_2007/msr_usr_1	73.6	80.4	376.8	391.2
msr_2007/msr_prn_1	90.4	91.5	351.7	390.4
msr_2007/msr_proj_4	110.0	113.5	425.1	563.9
msr_2007/msr_proj_1	118.6	121.3	447.0	505.6
meta_storage/block_traces_1	134.2	160.0	277.2	351.6
meta_storage/block_traces_2	137.9	179.4	300.5	374.9
msr_2007/msr_web_2	155.5	145.4	478.4	513.7
meta_storage/block_traces_4	156.2	179.2	371.8	384.8
meta_storage/block_traces_5	157.4	178.1	336.2	377.9
meta_storage/block_traces_3	158.4	178.6	312.7	380.4
msr_2007/msr_src1_0	219.3	200.8	628.9	588.3
msr_2007/msr_usr_2	222.3	215.6	627.7	579.9
msr_2007/msr_proj_2	252.3	265.0	528.3	527.5
msr_2007/msr_src1_1	274.7	274.2	652.3	633.4

Under concurrent cold-cache replay, SIEVE is 2–6x faster than the better of LRU/ARC on every trace. The margin is smaller than the warm table because every miss serializes through the same write mutex in all three caches — SIEVE's edge here comes entirely from its lock-free hit path. The best case is msr_prxy_0 (95% hits, 6.1x); the worst cases are the high-miss-ratio traces (msr_src1_1, 98% misses, 2.3x).

(An earlier revision of this table claimed 18–62x. That number was an artifact of all goroutines walking the trace from offset 0 in lockstep, which inflated hit ratios; the staggered-offset harness fixes it.)

Miss ratio (every trace)

Cache sized at 10% of unique keys. Bold = best in row.

Trace	SIEVE k=1	SIEVE k=2	SIEVE k=3	LRU	ARC
meta_storage/block_traces_1	0.4632	0.4651	0.4672	0.4602	0.4667
meta_storage/block_traces_2	0.4719	0.4743	0.4754	0.4676	0.4755
meta_storage/block_traces_3	0.4908	0.4928	0.4948	0.4885	0.4947
meta_storage/block_traces_4	0.4841	0.4870	0.4888	0.4812	0.4887
meta_storage/block_traces_5	0.4959	0.4984	0.4998	0.4927	0.5003
msr_2007/msr_hm_0	0.2991	0.3025	0.3025	0.3188	0.2923
msr_2007/msr_prn_0	0.2156	0.2194	0.2208	0.2310	0.2145
msr_2007/msr_prn_1	0.3908	0.3837	0.3796	0.4341	0.4148
msr_2007/msr_proj_0	0.2537	0.2660	0.2745	0.2375	0.2242
msr_2007/msr_proj_1	0.6794	0.6794	0.6794	0.7215	0.6788
msr_2007/msr_proj_2	0.8231	0.8231	0.8231	0.8548	0.8125
msr_2007/msr_proj_4	0.8463	0.8463	0.8463	0.8140	0.7173
msr_2007/msr_prxy_0	0.0512	0.0572	0.0594	0.0476	0.0468
msr_2007/msr_src1_0	0.7845	0.7845	0.7845	0.9132	0.7811
msr_2007/msr_src1_1	0.7939	0.7934	0.7934	0.8129	0.8231
msr_2007/msr_usr_1	0.3558	0.3558	0.3558	0.4007	0.3513
msr_2007/msr_usr_2	0.7216	0.7216	0.7216	0.7533	0.7199
msr_2007/msr_web_2	0.9786	0.9786	0.9786	0.9929	0.9785

Overall best (bold, computed by mktables): ARC has the lowest miss ratio in 11 of 18 rows, LRU in 5 (all meta_storage), SIEVE k-variants in 2 — msr_prn_1 (k=3 outright) and msr_src1_1 (k=2 and k=3 tied at 0.7934). SIEVE k=1 is never the overall best.

Head-to-head, SIEVE k=1 vs LRU (the typical deployment choice): SIEVE k=1 has lower miss ratio on 10 of 18 traces, LRU on 8. When SIEVE is better the margins are large (msr_src1_0: 12.9 points, msr_usr_1: 4.5, msr_prn_1: 4.3). When LRU is better the margins are narrow (all 5 meta_storage: 2–3 points each).

SIEVE's case rests on throughput. Miss ratios are competitive (SIEVE is never dramatically worse than LRU), and SIEVE is 2–6x faster on a cold mixed workload and 13–55x faster on warm reads, with the gap widening as cores are added. SIEVE k=3 produces the single-best entry in the entire table on msr_prn_1 (0.3796 vs LRU 0.4341, ARC 0.4148).

Memory during replay

On the 13.2M-request meta_storage/block_traces_1 trace (601K-entry cache), TestGCPressure reports:

Variant	TotalAlloc
SIEVE k=1	135 MB
SIEVE k=3	135 MB
LRU	418 MB
ARC	997 MB

SIEVE allocates 3.1x less than LRU and 7.4x less than ARC during replay — the array-backed node pool and inline-int32 xsync.MapOf are structural wins.

Full per-trace tables (sequential replay ns/op, B/op, miss ratio for every trace) are in bench-results.md. Methodology and trace-loading details are in bench/README.md.

Usage

import "github.com/opencoff/go-sieve"

// Create a cache mapping string keys to int values, capacity 1000.
c, err := sieve.New[string, int](1000)
if err != nil {
    log.Fatal(err) // ErrInvalidCapacity or ErrInvalidVisitClamp
}

// Or, for constant arguments, use Must to get a one-liner:
c := sieve.Must(sieve.New[string, int](1000))

c.Add("foo", 42)

if val, ok := c.Get("foo"); ok {
    fmt.Println(val) // 42
}

// Probe inserts only if absent; returns the cached value if present.
val, _, r := c.Probe("foo", 99)
// val == 42, r.Hit() == true

c.Delete("foo")
c.Purge() // reset entire cache

SIEVE-k

WithVisitClamp(k) creates a SIEVE-k cache where each entry uses a saturating counter instead of a single visited bit. An item accessed k+1 times survives k eviction passes before being evicted. k=1 is equivalent to classic SIEVE (the default). Use k=2 or k=3 for workloads with repeated access patterns where extra eviction resistance is beneficial.

// Classic SIEVE (k=1, the default)
c, err := sieve.New[string, int](1000)

// SIEVE-k=3: items survive up to 3 eviction passes
c, err := sieve.New[string, int](1000, sieve.WithVisitClamp(3))

New returns an error for capacity <= 0 (ErrInvalidCapacity) and for WithVisitClamp(k) with k > sieve.MaxVisitClamp (7 — ErrInvalidVisitClamp). Clamp values below 1 are silently rounded up to 1.

Eviction Handling

Add() and Probe() return the evicted entry (if any) along with a CacheResult bitmask. This allows callers to handle evictions synchronously without channels, goroutines, or lifecycle management.

c := sieve.Must(sieve.New[string, int](1000))

ev, r := c.Add("foo", 42)
if r.Evicted() {
    cleanupDisk(ev.Key, ev.Val)
}

// CacheResult bitmask:
//   CacheHit   — key was already present (value updated, no eviction)
//   CacheEvict — an entry was evicted to make room (mutually exclusive with CacheHit)

Purge() and Delete() do not report evictions.

Iteration

All() returns a Go 1.23+ iter.Seq2[K, V] over the cache contents. Iteration is weakly consistent and safe to call concurrently with Get/Add/Probe/Delete (including from inside the loop body):

for k, v := range c.All() {
    fmt.Println(k, v)
}

• Order is unspecified — it does not follow SIEVE FIFO order.
• Entries inserted during the walk may or may not be observed; entries evicted or deleted during the walk are skipped.
• Walking the cache does not mark entries as visited, so iteration does not protect entries from eviction (unlike Get).
• No cache lock is held across the loop body, so re-entrant Get/Add/Delete calls from inside range are safe.

API

Function / Method	Description
New[K, V](capacity, ...Option) (*Sieve[K,V], error)	Create a cache with fixed capacity. Returns ErrInvalidCapacity or ErrInvalidVisitClamp on bad input.
Must[K, V](*Sieve[K,V], error) *Sieve[K,V]	Helper that panics on error; useful with constant arguments.
Get(key) (V, bool)	Look up a key (lock-free, zero-alloc).
Add(key, val) (Evicted[K,V], CacheResult)	Insert or update; returns evicted entry and result bitmask.
Probe(key, val) (V, Evicted[K,V], CacheResult)	Insert-if-absent; returns cached/inserted value, evicted entry, and result bitmask.
Delete(key) bool	Remove a key.
Purge()	Clear the entire cache.
Len() int	Current number of entries (lock-free atomic load).
Cap() int	Maximum capacity.
All() iter.Seq2[K, V]	Weakly-consistent range iterator over current entries; does not mark visited.

Options

Option	Description
WithVisitClamp(k)	Use k-level saturating counters (default k=1 = classic SIEVE). k is capped at MaxVisitClamp (7); k < 1 is silently rounded to 1.

Key Constraints

The zero value of K is reserved — do not use it as a key ("" for string keys, 0 for integer keys, the zero struct for struct keys).

This is by design: when an entry is evicted or deleted, its slot's key is zeroed in place, and the lock-free read path relies on a zeroed key never matching a live lookup. If you store the zero key, a Get racing with that key's eviction can observe the zeroed slot and return a spurious hit with a zero value. In single-threaded use the zero key appears to work, but the contract is: keys must be non-zero.

If your natural key space includes the zero value, wrap it (e.g. offset integer keys by 1, or prefix string keys).

GC Note

When K or V is a pointer type (including string, which contains an internal pointer in Go), the node array will still contain GC-traced pointers. The GC pressure reduction is most dramatic for scalar key/value types (int, [16]byte, fixed-size structs).

License

BSD-2-Clause. See the source files for the full license text.