Skip to content

Add docs for src/unix/raw.mli - #380

Closed
tomjridge wants to merge 1 commit into
mirage:mainfrom
tomjridge:tjr@add_documentation
Closed

tomjridge wants to merge 1 commit into
mirage:mainfrom
tomjridge:tjr@add_documentation

Conversation

@tomjridge

Copy link
Copy Markdown
Contributor

No description provided.

@Ngoguey42 Ngoguey42 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for documenting all this :-)

Can you pass ocamlformat so that we can make that PR mergeable?

Comment thread src/unix/raw.ml
Comment on lines +21 to +22
(* NOTE the return value is the number of bytes read; this is not completely obvious from
the code *)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The returned value is the new offset, after the values read

Comment thread src/unix/raw.mli
Comment on lines +15 to +16
the file data (see irmin-pack's [Version] module - were the way to encode objects was
changed from [`V1] to [`V2]; [Version.set t s] will use exactly the first 8 bytes of the

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
the file data (see irmin-pack's [Version] module - were the way to encode objects was
changed from [`V1] to [`V2]; [Version.set t s] will use exactly the first 8 bytes of the
the file data (see irmin-pack's [Version] module - where the way to encode objects was
changed from [`V1] to [`V2]); [Version.set t s] will use exactly the first 8 bytes of the

Comment thread src/unix/raw.mli
Comment on lines +27 to +30
How does the user know where the header ends and the file data begins? One option is to
call {!fstat} after initializing the header info. In fact, in {!Index_unix} there is a
separate computation of the header size: [let header = eight ++ eight ++ eight ++ eight ++
fan_size]; presumably other users of this module do the same.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread src/unix/raw.mli
(** An exception that can be raised when {b reading} (not writing!) header fields, in case
the amount of bytes requested for a field is not the amount of bytes actually read
from file. This presumably happens only for files that are not initialized correctly,
i.e., they do not contain a full header. *)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can happen in several other situations

Suggested change
i.e., they do not contain a full header. *)
e.g., they do not contain a full header. *)

Comment thread src/unix/raw.mli
the amount of bytes requested for a field is not the amount of bytes actually read
from file. This presumably happens only for files that are not initialized correctly,
i.e., they do not contain a full header. *)
(* FIXME rename this exception? *)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that documenting it is enough

@balat balat mentioned this pull request Nov 9, 2026
@balat

balat commented Nov 13, 2026

Copy link
Copy Markdown
Member

Thanks @tomjridge! This documentation is included in #406 (commit ba46e56), with a few corrections and additions, and will be part of 1.7.0.

@balat balat closed this Nov 13, 2026
jmid pushed a commit to ocaml/opam-repository that referenced this pull request Nov 17, 2026
CHANGES:

## Added

- Add `Stats.merge` and `Stats.merge_in`, to combine the statistics recorded
  by several domains (mirage/index#404, @lyrm)
- Expose the Unix platform as `Index_unix.Private.Platform`, so that
  `Index.Make` can be applied to it directly (mirage/index#396, @art-w)
- Document the file format of `Index_unix.Private.Raw` (mirage/index#380, @tomjridge,
  @balat)

## Changed

- Breaking: OCaml 5.0 or later is now required (mirage/index#404, @lyrm)
- Breaking: statistics are now recorded per domain, in a `Domain.DLS` key,
  instead of in a global value. `Stats.get` and `Stats.reset_stats` only see
  the calling domain (mirage/index#404, @lyrm)
- Mirage support: `index` no longer depends on `unix`, and `index.unix` is
  now an optional library (mirage/index#396, @art-w)
- Breaking: `Platform.S` and `Checks.Platform_args` now require a `Progress`
  and a `Fmt_tty` module, and `Private.Logs.setup` and
  `Private.Logs.setup_term` take an extra `(module Platform.FMT_TTY)`
  argument (mirage/index#396, @art-w)
- Support cmdliner 2.0.0. Cmdliner 1.1.0 or later is now required (mirage/index#382,
  @MisterDA, mirage/index#402, @art-w)

## Fixed

- Lock files are now opened with the `O_CLOEXEC` flag (mirage/index#394, @vect0r-vicall)
- Read-only instances could return stale values after `sync`. `sync` now
  removes from their LRU the keys of the entries it loads, from the log or
  from the log used during merges. It also holds the lock that `find` takes,
  so that a concurrent `find` cannot cache an outdated value in the middle of
  a `sync` (mirage/index#398, mirage/index#399, @art-w, mirage/index#406, @balat)
- Merges could lose entries, and `iter` could give wrong keys, when values
  are encoded on zero bytes and the key decoder returns its input string
  without copying it, as `Repr.decode_bin` does (mirage/index#387, @Ngoguey42)
- Concurrent `find`s on distinct instances of the same `Index.Make`
  application, from several threads or domains, could miss bindings: they
  shared the buffer in which they prefetch entries of the data file. Each
  instance now has its own buffer, created with
  `Private.Io_array.prefetch_buffer` and passed to `Private.Io_array.v`
  (mirage/index#406, @balat)
- `Index_unix.Private.Raw`: reading a truncated header now raises
  `Not_written`, as for a missing header, instead of failing an assertion.
  Writing a version that is not 8 bytes long raises `Invalid_argument`, and a
  write that makes no progress raises `Failure` instead of being dropped
  (mirage/index#406, @balat)
- Add the missing `logs.threaded` dependency of the Unix tests (mirage/index#400,
  @anmonteiro)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants