Report an error when pushing conflicting metadata.#73
Open
plietar wants to merge 1 commit into
Open
Conversation
Our packet IDs are generally assumed to be globally unique, but this is not guaranteed by construction: the ID is not a hash of its contents, and the metadata store is not content-addressed. This means one could try to upload two different packets to the store with the same ID. Previously, outpack server would accept the second request but skip writing to the file. It is better to detect this situation and to return an HTTP error when it occurs.
plietar
commented
Feb 11, 2025
| /// | ||
| /// Succeeds if the file already exists with identical contents. | ||
| /// On the other hand, an AlreadyExists error is returned if the file exists with different contents. | ||
| pub fn write_file_idempotent(path: &Path, contents: &[u8]) -> io::Result<()> { |
Member
Author
There was a problem hiding this comment.
I don't love this name, but I also couldn't find a better one that expresses the idea of detecting conflicts.
richfitz
reviewed
Feb 11, 2025
Comment on lines
+39
to
+41
| if fs::read(path)? != contents { | ||
| return Err(err); | ||
| } |
Member
There was a problem hiding this comment.
Are we ok with end-of-line standardisartion between windows/everyone else here? I can't remember what we do in this case - perhaps we dodge it by spitting everything out on one line first?
Member
Author
There was a problem hiding this comment.
outpack_server doesn't ever do any special processing of newline or anything else. Metadata files are an immutable blob of bytes.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Our packet IDs are generally assumed to be globally unique, but this is not guaranteed by construction: the ID is not a hash of its contents, and the metadata store is not content-addressed.
This means one could try to upload two different packets to the store with the same ID. Previously, outpack server would accept the second request but skip writing to the file. It is better to detect this situation and to return an HTTP error when it occurs.