Add S3FIFO Eviction Algorithm

[williamnixon20]

This PR adds the S3FIFO eviction algorithm, as described here.

Running cachebench results show that S3FIFO achieves the highest hit rate among all evaluated algorithms. Throughput wise, it performs similarly with TinyLFU.

Our implementation adapts the existing TinyLFUMM.

Algorithm Detail
The algorithm maintains three structures:

• S — a small queue for newly inserted items (10% of size)
• M — a main FIFO queue (90% of size)
• G — a ghost queue for items evicted from S (size of M), only tracks the object key hash.

The general idea of the algorithm is:

• Evict early: items in S are aggressively filtered to not pollute the main cache.
• Promote late: promotion/reinsertion occurs only when an accessed item reaches the tail.

Each object carries one access bit that records whether it has been accessed since it was inserted or last inspected.

Items in the tail are:

• If accessed and in S, moved to M.
• If accessed and in M, reinserted to head of M.
• If not accessed, evict. If this item is evicted from S, move it to the ghost queue.

Insertions are done as such:

• If the item exists in ghost, directly insert to M.
• Else insert to S.

We try to do eviction from S if the size > expected size (10%). Else, we try to evict from main.

Eviction iterator starts from tail and skips accessed items. Promotions (reinsertions) are done 1x before iterator is created.

Benchmark:
We evaluated S3FIFO using the test configurations in test_configs/hit_ratio with 48 threads and an 8 GB cache. We compared this against other existing MMContainers(LRU2Q, LRU, and TinyLFU). We use the DebugRel release version for testing.

The results can be found here
S3 FIFO Benchmark (2).pdf

Miss ratio:
S3FIFO consistently outperforms LRU, LRU2Q, and TinyLFU across all workloads.

Throughput:
Throughput is comparable to TinyLFU.

This implementation is not a throughput optimized version and still heavily uses LRU locks (in this case, TinyLFU's spinlocks). There maybe room for further improvement.

Notes:

• A FIFOHashSet is implemented to act as a ghost queue, using deque for maintaining FIFO order and unordered set for membership.
• Several unused LRU configurations were removed as part of cleanup.

The parameters and the defaults are:
(1) Small / probationary queue size (default 10%)
Controls how long new objects stay in cache. If the workload has many low-reuse items (objects that appear only once or only a few times), making this smaller helps avoid pollution because these items get evicted quickly without entering the main cache.
(2) Large queue size (default 90%)
The rest of the cache. This is the “main” queue that holds objects with higher reuse.
(3) Ghost queue size (default 90%)
Stores only the key hashes of items evicted from the small queue. A larger ghost helps catch objects that return shortly after eviction. If the workload has many items that appear only twice and never come back again, the ghost queue can be made smaller since promoting these into the main queue doesn’t help.

For tuning, the paper shows that the small queue generally works best between 5% and 20% so those values can be tried out. If we have the proportion of the object reuse in the trace it can help us determine the sizes too.

[meta-cla]

Hi @williamnixon20!

Thank you for your pull request and welcome to our community.

Action Required

In order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you.

Process

In order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA.

Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with CLA signed. The tagging process may take up to 1 hour after signing. Please give it that time before contacting us about it.

If you have received this in error or have any questions, please contact us at [email protected]. Thanks!

[meta-cla]

Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Meta Open Source project. Thanks!

[meta-codesync]

@pbhandar2 has imported this pull request. If you are a Meta employee, you can view this in D87821750.

[facebook-github-bot]

@williamnixon20 has updated the pull request. You must reimport the pull request before landing.

[facebook-github-bot]

@williamnixon20 has updated the pull request. You must reimport the pull request before landing.

[pbhandar2]

@williamnixon20 Thank you so much for the PR! S3FIFO would be a great replacement policy to offer.

Few comments:

• Add tests for MMS3FIFO? Please add MMS3FIFO to any typed test where it is relevant and also add MMS3FIFO specific tests to validate its behavior. Maybe add a MMS3Allocator type to allocator type tests too?
• The automated checks are reporting linter errors. It would be great if you could fix those.
• Add CacheAllocatorS3FIFO5BCache.cpp and the related definition of CacheTrait that uses CompressedPtr5B.

[pbhandar2]

@williamnixon20 Could you also describe each configuration parameter of S3FIFO, the default values, when and how to tune the them in the summary?

[williamnixon20]

@williamnixon20 Could you also describe each configuration parameter of S3FIFO, the default values, when and how to tune the them in the summary?

The parameters and the defaults are:
(1) Small / probationary queue size (default 10%)
Controls how long new objects stay in cache. If the workload has many low-reuse items (objects that appear only once or only a few times), making this smaller helps avoid pollution because these items get evicted quickly without entering the main cache.
(2) Large queue size (default 90%)
The rest of the cache. This is the “main” queue that holds objects with higher reuse.
(3) Ghost queue size (default 90%)
Stores only the key hashes of items evicted from the small queue. A larger ghost helps catch objects that return shortly after eviction. If the workload has many items that appear only twice and never come back again, the ghost queue can be made smaller since promoting these into the main queue doesn’t help.

For tuning, the paper shows that the small queue generally works best between 5% and 20% so those values can be tried out. If we have the proportion of the object reuse in the trace it can help us determine the sizes too.

[pbhandar2]

@williamnixon20 Could you also describe each configuration parameter of S3FIFO, the default values, when and how to tune the them in the summary?

The parameters and the defaults are: (1) Small / probationary queue size (default 10%) Controls how long new objects stay in cache. If the workload has many low-reuse items (objects that appear only once or only a few times), making this smaller helps avoid pollution because these items get evicted quickly without entering the main cache. (2) Large queue size (default 90%) The rest of the cache. This is the “main” queue that holds objects with higher reuse. (3) Ghost queue size (default 90%) Stores only the key hashes of items evicted from the small queue. A larger ghost helps catch objects that return shortly after eviction. If the workload has many items that appear only twice and never come back again, the ghost queue can be made smaller since promoting these into the main queue doesn’t help.

For tuning, the paper shows that the small queue generally works best between 5% and 20% so those values can be tried out. If we have the proportion of the object reuse in the trace it can help us determine the sizes too.

@williamnixon20 Please add it to the PR summary if you able to edit it or I will add it later.

[facebook-github-bot]

@williamnixon20 has updated the pull request. You must reimport the pull request before landing.

[facebook-github-bot]

@williamnixon20 has updated the pull request. You must reimport the pull request before landing.

[facebook-github-bot]

@williamnixon20 has updated the pull request. You must reimport the pull request before landing.

[facebook-github-bot]

@williamnixon20 has updated the pull request. You must reimport the pull request before landing.

[facebook-github-bot]

@williamnixon20 has updated the pull request. You must reimport the pull request before landing.

[facebook-github-bot]

@williamnixon20 has updated the pull request. You must reimport the pull request before landing.

[facebook-github-bot]

@williamnixon20 has updated the pull request. You must reimport the pull request before landing.

[facebook-github-bot]

@williamnixon20 has updated the pull request. You must reimport the pull request before landing.