Skip to content

Fix RO sync: clear LRU for all log updates - #399

Closed
art-w wants to merge 2 commits into
mirage:mainfrom
art-w:log-sync
Closed

art-w wants to merge 2 commits into
mirage:mainfrom
art-w:log-sync

Conversation

@art-w

@art-w art-w commented May 24, 2024

Copy link
Copy Markdown
Contributor

As reported by @edwintorok in #398, a read-only instance would sometimes fail to find the latest value after doing a sync, because it would keep finding the old value in its LRU cache (and ignore the more recent writes).

The quick-fix is to always clear the LRU whenever sync discovers that the log has changed on disk... It's not super satisfying as we could be more fine grained and update the LRU with the new bindings from the log, but it's the strategy the code was already using for some (but not all) log updates.

@edwintorok

Copy link
Copy Markdown

Clearing all entries is probably safer indeed. Another approach might be to have a generation counter on each entry, and then compare the generation counter of the file with the generation counter of the cached entry, and consider the entry stale if it is older.

@art-w

art-w commented May 27, 2024

Copy link
Copy Markdown
Contributor Author

Hmm I'm slightly worried of the performance impact of double-checking the LRU freshness on every query, it felt more predictable to only penalize the read-only instance for calling sync by clearing its LRU. Wouldn't the generation counter be equivalent to an LRU clear as it would invalidate all entries that are older than the latest update? (note that we only call Lru.clear if sync sees that an update happened, but otherwise the LRU is preserved)

That being said, I believe it should be possible for sync to update the LRU with the newer values read from the log, but the code change is more involved... Perhaps as a follow-up PR, unless you are already seeing poor performances caused by the LRU clear?

@art-w
art-w requested a review from clecat May 27, 2024 08:52
@edwintorok

Copy link
Copy Markdown

Hmm I'm slightly worried of the performance impact of double-checking the LRU freshness on every query, it felt more predictable to only penalize the read-only instance for calling sync by clearing its LRU. Wouldn't the generation counter be equivalent to an LRU clear as it would invalidate all entries that are older than the latest update? (note that we only call Lru.clear if sync sees that an update happened, but otherwise the LRU is preserved)

You are right, keeping a RO client efficiently in-sync with another one is not really possible currently, and the generation counter wouldn't really help (you need to discard all LRU entries, whenever anything changes in the index, even if we have per-entry generation counters, we'd still need to read that counter to know whether it changed or not).

That being said, I believe it should be possible for sync to update the LRU with the newer values read from the log, but the code change is more involved... Perhaps as a follow-up PR, unless you are already seeing poor performances caused by the LRU clear?

Yes, that sounds like a better approach than what I was suggesting with the generation counters. Followup PR sounds good, currently I'm just evaluating whether I can use 'index' in a project, I'm not using it yet as such.

@clecat
clecat removed their request for review June 21, 2026 14:13
@balat balat mentioned this pull request Oct 9, 2026
@balat

balat commented Nov 13, 2026

Copy link
Copy Markdown
Member

Thanks @art-w! This fix is included in #406 (commit a2e885d), which is now merged and will be part of 1.7.0.

@balat balat closed this Nov 13, 2026
jmid pushed a commit to ocaml/opam-repository that referenced this pull request Nov 17, 2026
CHANGES:

## Added

- Add `Stats.merge` and `Stats.merge_in`, to combine the statistics recorded
  by several domains (mirage/index#404, @lyrm)
- Expose the Unix platform as `Index_unix.Private.Platform`, so that
  `Index.Make` can be applied to it directly (mirage/index#396, @art-w)
- Document the file format of `Index_unix.Private.Raw` (mirage/index#380, @tomjridge,
  @balat)

## Changed

- Breaking: OCaml 5.0 or later is now required (mirage/index#404, @lyrm)
- Breaking: statistics are now recorded per domain, in a `Domain.DLS` key,
  instead of in a global value. `Stats.get` and `Stats.reset_stats` only see
  the calling domain (mirage/index#404, @lyrm)
- Mirage support: `index` no longer depends on `unix`, and `index.unix` is
  now an optional library (mirage/index#396, @art-w)
- Breaking: `Platform.S` and `Checks.Platform_args` now require a `Progress`
  and a `Fmt_tty` module, and `Private.Logs.setup` and
  `Private.Logs.setup_term` take an extra `(module Platform.FMT_TTY)`
  argument (mirage/index#396, @art-w)
- Support cmdliner 2.0.0. Cmdliner 1.1.0 or later is now required (mirage/index#382,
  @MisterDA, mirage/index#402, @art-w)

## Fixed

- Lock files are now opened with the `O_CLOEXEC` flag (mirage/index#394, @vect0r-vicall)
- Read-only instances could return stale values after `sync`. `sync` now
  removes from their LRU the keys of the entries it loads, from the log or
  from the log used during merges. It also holds the lock that `find` takes,
  so that a concurrent `find` cannot cache an outdated value in the middle of
  a `sync` (mirage/index#398, mirage/index#399, @art-w, mirage/index#406, @balat)
- Merges could lose entries, and `iter` could give wrong keys, when values
  are encoded on zero bytes and the key decoder returns its input string
  without copying it, as `Repr.decode_bin` does (mirage/index#387, @Ngoguey42)
- Concurrent `find`s on distinct instances of the same `Index.Make`
  application, from several threads or domains, could miss bindings: they
  shared the buffer in which they prefetch entries of the data file. Each
  instance now has its own buffer, created with
  `Private.Io_array.prefetch_buffer` and passed to `Private.Io_array.v`
  (mirage/index#406, @balat)
- `Index_unix.Private.Raw`: reading a truncated header now raises
  `Not_written`, as for a missing header, instead of failing an assertion.
  Writing a version that is not 8 bytes long raises `Invalid_argument`, and a
  write that makes no progress raises `Failure` instead of being dropped
  (mirage/index#406, @balat)
- Add the missing `logs.threaded` dependency of the Unix tests (mirage/index#400,
  @anmonteiro)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants