Skip to content

Replace LRU/MRU/FIFO/LFU with SIEVE eviction#20

Merged
toloco merged 4 commits intomasterfrom
sieve-eviction
Mar 6, 2026
Merged

Replace LRU/MRU/FIFO/LFU with SIEVE eviction#20
toloco merged 4 commits intomasterfrom
sieve-eviction

Conversation

@toloco
Copy link
Owner

@toloco toloco commented Mar 5, 2026

Summary

  • Unified eviction: Both in-process (memory) and cross-process (shared) backends now use SIEVE — a simple, scan-resistant algorithm with near-optimal hit rates
  • Lock-free shared reads: All shared memory reads are now fully lock-free (previously only FIFO was lock-free; LRU/MRU/LFU required a write lock on every hit for ordering updates). SIEVE's visited bit is set via an idempotent store with no lock
  • Simplified API: Removed the strategy parameter and Strategy enum — cache() now just works with SIEVE, no configuration needed
  • 540 fewer lines: Deleted src/strategies/ module (5 files), simplified ordering.rs from multi-strategy dispatch to a single sieve_evict() function
  • Layout VERSION 2→3: Forces recreation of old mmap files (field renames: _reservedsieve_hand, frequencyvisited)

Files changed (27 files, -540 net lines)

Area Files Change
Shared memory core layout.rs, ordering.rs, mod.rs, region.rs SIEVE eviction, lock-free reads
Memory core store.rs, entry.rs, strategies/ (deleted) Inline SIEVE, remove strategy dispatch
Python API _decorator.py, _strategies.py, __init__.py, .pyi Remove strategy param and Strategy enum
Tests 8 test files Remove strategy params, add SIEVE-specific tests

Test plan

  • cargo clippy -- -D warnings — no warnings
  • ruff check + ty check — all clean
  • pytest tests/ -v — all 72 tests pass
  • SIEVE eviction order verified: unvisited entries evicted first, visited entries get second chance
  • Shared memory reads are fully lock-free (no write_lock() in get path)
  • Cross-process tests pass (visibility, concurrent writers, hashseed independence)

🤖 Generated with Claude Code

toloco and others added 3 commits March 5, 2026 11:45
Unify the eviction algorithm to SIEVE for both the in-process memory
backend and the cross-process shared memory backend. This simplifies
the codebase (removes ~540 lines), eliminates the `strategy` parameter
and `Strategy` enum, and makes shared memory reads fully lock-free
(visited bit is set via an idempotent store, no write lock needed).

Key changes:
- Shared memory: replace multi-strategy dispatch with sieve_evict()
- Layout VERSION bumped 2→3 (forces recreation of old mmap files)
- All reads on shared backend are now lock-free (previously only FIFO)
- Remove src/strategies/ module entirely (memory backend uses inline SIEVE)
- Remove Strategy enum from Python API; cache() no longer takes strategy=

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace all references to LRU/MRU/FIFO/LFU strategy selection with
SIEVE eviction across README, usage guide, performance docs, alternatives
comparison, benchmarks, llms.txt, examples, and CLAUDE.md.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Run benchmarks after SIEVE migration to capture improvements:
- Shared backend: 7.8M → 8.9M ops/s (lock-free reads via visited bit)
- Multi-process 4 procs: 3.1M → 7.7M ops/s (2.5x improvement)
- Multi-process 8 procs: 1.9M → 6.5M ops/s (3.4x improvement)
- Hit rate: 64.9% → 72.7% (SIEVE scan-resistant eviction)

Also fix bench_runner SharedCachedFunction calls (removed old strategy arg).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@toloco toloco merged commit 2aedb6b into master Mar 6, 2026
14 checks passed
@toloco toloco deleted the sieve-eviction branch March 6, 2026 10:40
@toloco toloco mentioned this pull request Mar 6, 2026
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant