Describe the bug
Duplicate detection is intended to read only selected bytes for files larger than 16 KiB, but the threshold is defined as int64(16 * KiB). In this codebase KiB is a SizeUnit value based on internal display-unit math, not a byte count.
This makes the threshold much larger than 16 KiB, so fileHash can read whole large files with os.ReadFile, which is risky for memory usage and performance.
To Reproduce
- Inspect
internal/content/duplicate.go:
var thresholdFileSize = int64(16 * KiB)
- Compare with
internal/content/size.go, where KiB is not the byte value 1024.
- Run duplicate detection over large files and observe that files below the inflated threshold take the whole-file path.
Screenshots
Not applicable.
Info (please complete the following information):
Additional context
The threshold should likely be a byte count such as 16 * 1024 or use an explicit conversion helper that returns bytes.
Describe the bug
Duplicate detection is intended to read only selected bytes for files larger than 16 KiB, but the threshold is defined as
int64(16 * KiB). In this codebaseKiBis aSizeUnitvalue based on internal display-unit math, not a byte count.This makes the threshold much larger than 16 KiB, so
fileHashcan read whole large files withos.ReadFile, which is risky for memory usage and performance.To Reproduce
internal/content/duplicate.go:internal/content/size.go, whereKiBis not the byte value1024.Screenshots
Not applicable.
Info (please complete the following information):
Additional context
The threshold should likely be a byte count such as
16 * 1024or use an explicit conversion helper that returns bytes.