Repository navigation
Conversation
|
Clearing all entries is probably safer indeed. Another approach might be to have a generation counter on each entry, and then compare the generation counter of the file with the generation counter of the cached entry, and consider the entry stale if it is older. |
|
Hmm I'm slightly worried of the performance impact of double-checking the LRU freshness on every query, it felt more predictable to only penalize the read-only instance for calling That being said, I believe it should be possible for |
You are right, keeping a RO client efficiently in-sync with another one is not really possible currently, and the generation counter wouldn't really help (you need to discard all LRU entries, whenever anything changes in the index, even if we have per-entry generation counters, we'd still need to read that counter to know whether it changed or not).
Yes, that sounds like a better approach than what I was suggesting with the generation counters. Followup PR sounds good, currently I'm just evaluating whether I can use 'index' in a project, I'm not using it yet as such. |
CHANGES: ## Added - Add `Stats.merge` and `Stats.merge_in`, to combine the statistics recorded by several domains (mirage/index#404, @lyrm) - Expose the Unix platform as `Index_unix.Private.Platform`, so that `Index.Make` can be applied to it directly (mirage/index#396, @art-w) - Document the file format of `Index_unix.Private.Raw` (mirage/index#380, @tomjridge, @balat) ## Changed - Breaking: OCaml 5.0 or later is now required (mirage/index#404, @lyrm) - Breaking: statistics are now recorded per domain, in a `Domain.DLS` key, instead of in a global value. `Stats.get` and `Stats.reset_stats` only see the calling domain (mirage/index#404, @lyrm) - Mirage support: `index` no longer depends on `unix`, and `index.unix` is now an optional library (mirage/index#396, @art-w) - Breaking: `Platform.S` and `Checks.Platform_args` now require a `Progress` and a `Fmt_tty` module, and `Private.Logs.setup` and `Private.Logs.setup_term` take an extra `(module Platform.FMT_TTY)` argument (mirage/index#396, @art-w) - Support cmdliner 2.0.0. Cmdliner 1.1.0 or later is now required (mirage/index#382, @MisterDA, mirage/index#402, @art-w) ## Fixed - Lock files are now opened with the `O_CLOEXEC` flag (mirage/index#394, @vect0r-vicall) - Read-only instances could return stale values after `sync`. `sync` now removes from their LRU the keys of the entries it loads, from the log or from the log used during merges. It also holds the lock that `find` takes, so that a concurrent `find` cannot cache an outdated value in the middle of a `sync` (mirage/index#398, mirage/index#399, @art-w, mirage/index#406, @balat) - Merges could lose entries, and `iter` could give wrong keys, when values are encoded on zero bytes and the key decoder returns its input string without copying it, as `Repr.decode_bin` does (mirage/index#387, @Ngoguey42) - Concurrent `find`s on distinct instances of the same `Index.Make` application, from several threads or domains, could miss bindings: they shared the buffer in which they prefetch entries of the data file. Each instance now has its own buffer, created with `Private.Io_array.prefetch_buffer` and passed to `Private.Io_array.v` (mirage/index#406, @balat) - `Index_unix.Private.Raw`: reading a truncated header now raises `Not_written`, as for a missing header, instead of failing an assertion. Writing a version that is not 8 bytes long raises `Invalid_argument`, and a write that makes no progress raises `Failure` instead of being dropped (mirage/index#406, @balat) - Add the missing `logs.threaded` dependency of the Unix tests (mirage/index#400, @anmonteiro)
As reported by @edwintorok in #398, a read-only instance would sometimes fail to find the latest value after doing a
sync, because it would keep finding the old value in its LRU cache (and ignore the more recent writes).The quick-fix is to always clear the LRU whenever
syncdiscovers that the log has changed on disk... It's not super satisfying as we could be more fine grained and update the LRU with the new bindings from the log, but it's the strategy the code was already using for some (but not all) log updates.