llamafile_open_zip: validate the zip entry window against the actual file size - #1039
llamafile_open_zip: validate the zip entry window against the actual file size#1039professor-moody wants to merge 3 commits into
Conversation
…file size get_zip_cfile_offset() and get_zip_cfile_compressed_size() both return int64_t and use -1 to signal failure, for example when the compressed size field is the zip64 sentinel 0xFFFFFFFF and no zip64 extra field is present. Both results were assigned straight into uint64_t and size_t, so a crafted archive could set file->size to SIZE_MAX. file->mapsize = skew + file->size then wrapped, mmap succeeded with a single page, and llamafile_size() reported SIZE_MAX from then on. There was also no check that the declared offset and size fit inside the file, so an ordinary oversized size field mapped past EOF without needing the sentinel at all. Three changes: - keep the real file length from the lseek already done at open time, since file->size holds it only until the archive's claim overwrites it - take both accessor results as int64_t, reject negatives before they reach unsigned types, and require the window to fit inside the file, written as a subtraction so it cannot overflow - require a nonzero central directory size in both the zip64 and zip32 branches; zero passed the existing <= INT_MAX test and led to malloc(0) followed by a 4 byte read
|
Thank you @professor-moody! I am trying to wrap up a release by EOD, so I will not be able to merge this immediately as I want to run the whole suite of integration tests on this. But I will test, review, and merge your PR afterwards, so we'll have time to make sure it satisfies all tests before the new release. I hope that's ok! 🙏 |
…le header The previous commit bounded the offset and size taken from the central directory, but that offset points at the local file header, not at the data. The existing off += ZIP_LFILE_HDRSIZE(lfile) then advances past namesize and extrasize, two attacker-controlled uint16 fields worth up to 131100 bytes, and nothing revalidated the window afterwards. An archive with an entirely honest central directory could therefore still push the mapping past the end of the file and fault on the first read. Revalidate after the advance, using a subtraction so it cannot overflow. Also require the central directory size to be at least kZipCfileHdrMinSize rather than merely nonzero: a record is 46 bytes by construction, and sizes of 1 to 3 otherwise reached malloc() followed by a 4 byte read. Checked against archives with a legitimate 8000 byte local extra field and against a real llamafile, both of which load unchanged.
|
That timing works fine, thanks for letting me know. Good thing you had not merged yet. I kept testing my own patch against a wider set of crafted archives while waiting, and found a gap in it. Pushed a fixup, so please review the branch rather than the original commit. The original bound validated the offset and size taken from the central directory. The problem is that offset points at the local file header, not at the data. The existing Two 182 byte archives with byte-identical central directories, running the guard as it stood: The fixup revalidates after the advance, written as a subtraction so it cannot overflow. It also tightens the central directory size check from I specifically checked that it does not over-reject, since that would be the worse failure for you: an archive with a legitimate 8000 byte local extra field still loads normally, as does a real llamafile. A large local extra field is legal and zipalign relies on it, so the fix bounds the resulting window rather than capping the header size. Worth noting for your test run: the original commit does still fix the more serious half on its own, since |
|
The ZIP64 branch still only checks ZIP_CDIR64_SIZE(bufdata) > 0, while ZIP32 now requires >= kZipCfileHdrMinSize. That seems to leave the small-central-directory OOB read open for ZIP64 too. I'd mirror the same minimum-size check there. |
… the zip64 branch
The previous commit required ZIP_CDIR_SIZE >= kZipCfileHdrMinSize on the zip32
branch but left the zip64 branch checking only ZIP_CDIR64_SIZE > 0. The two
branches should agree.
cdirsize is taken from that field and cdirdata = gc(malloc(cdirsize)) allocates
exactly that many bytes, after which llamafile.c:158 does
if (ZIP_READ32(cdirdata) != kZipCfileHdrMagic)
unconditionally. A declared size of 1, 2 or 3 reaches that four-byte read with a
shorter allocation.
Sizes 4 through 45 are not exposed to this residual: the read is in bounds, and
the entry loop short-circuits on
entry_offset + kZipCfileHdrMinSize <= cdirsize
at :171 before ZIP_CFILE_HDRSIZE is evaluated.
Requiring >= kZipCfileHdrMinSize closes the 1-3 case and makes the two branches
consistent.
Reported in review by tahazarif10.
|
@tahazarif10 Good catch, you're right — the two branches should agree. Tracing it against the branch head, the exposure is narrower than it first looks, so worth being precise. I swept the declared size on a build of this branch to confirm which path each one takes. No crash in any case, and I'm not claiming one — the over-read is small and I don't have a sanitised build to measure its extent, so that part is from reading the source rather than observing it. Pushed a commit applying the minimum-size guard to the zip64 branch, so it now matches zip32. |
|
@aittalam This is ready for the integration-test pass whenever it suits you. The branch now carries the original entry-window validation plus the zip64 minimum-size guard from @tahazarif10's review. |
Fixes the crash reported in #1038.
llamafile_open_zip() trusted the offset and compressed size from the zip central directory without checking them against the real file, and assigned the int64_t results of get_zip_cfile_offset() and get_zip_cfile_compressed_size() into uint64_t and size_t. Those functions return -1 on failure, so a crafted archive could set file->size to SIZE_MAX, wrap the mapsize arithmetic, and end up with a one page mapping that llamafile_size() then reported as SIZE_MAX bytes.
Three changes, all in llamafile/llamafile.c:
Keep the real file length from the lseek that already happens at open time. file->size holds it briefly before being overwritten at :178 by the value the archive claims, so this just preserves it as a local to validate against.
Take both accessor results as int64_t and reject negatives before they reach unsigned types, then require the declared window to fit inside the file. The bound is written as "entry_off > zipsize - entry_size" so it cannot overflow.
Require a nonzero central directory size in both the zip64 and zip32 branches. Zero passed the existing "<= INT_MAX" test and led to malloc(0) followed by a 4 byte read.
Tested against a build of 9a716e7 on Fedora 43 x86_64 and on arm64 macOS. Before the change the crafted 162 byte archive gives SIGSEGV on Linux and SIGBUS on macOS, 10 runs out of 10. After it, both are rejected with "zip entry has an unreadable offset or size", 0 out of 10. A separate archive using a plain oversized size field rather than the zip64 sentinel is rejected with "zip entry extends past the end of the file". Control archives behave exactly as before.
Regression checked against a real model. I packed a Qwen2.5-0.5B-Instruct Q4_K_M gguf into a llamafile with zipalign -j0 and ran it, so the load goes through llamafile_open_zip with legitimate values. Before and after the change it loads and generates byte-identical output. The loose-file path (-m on a plain .gguf) is unchanged too.