Keep borrowed packs open after cache eviction - #2349
Draft
bojanz wants to merge 3 commits into
Draft
Conversation
Contributor
Author
|
The Windows failures were real: Unsure how useful |
bojanz
force-pushed
the
pack-cache-release-on-eviction
branch
from
August 10, 2026 16:22
f73b999 to
f9dfba2
Compare
Turn resource-backed Pack iterators into generators that retain the owning pack. Add regression coverage for cache-released packs across all iterator APIs.
Contributor
Author
|
The last commit could be a followup PR as well if you prefer it that way. One behavioral note: work and errors that previously happened at the call, most notably the full sort in |
bojanz
marked this pull request as draft
August 27, 2026 21:35
Contributor
Author
|
Drafted cause I want to take another look at this once we land #2334 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Several code paths close a
Packat the moment it leaves the object-store cache:_add_cached_packcloses the pack it replaces,_enforce_packed_git_limitcloses the least-recently-used pack,_remove_packcloses on removal, and_update_pack_cachecloses packs whose files disappeared.Closing at eviction time is unsafe when the store is shared:
packsandget_rawhand out references to cachedPackobjects, so another consumer — a concurrent request in a threaded/greenlet server, or an iterator likeiter_unpackedthat is still mid-pack — may be reading from the pack when eviction closes it under its feet. The reader then fails on a closed file even though the pack file itself is still perfectly valid on disk.This changes eviction to release ownership instead of closing: the pack is dropped from the cache, and the underlying files are closed by
Pack.__del__once the last borrower is done with it. Since that hand-off is intentional,__del__no longer emits the "unclosed Pack"ResourceWarningfor packs released this way — explicit leaks still warn as before.The new test exercises the LRU-limit path: a pack evicted by
packed_git_limitwhile aniter_unpackediterator is mid-stream stays readable until the iterator completes.