Feature request: Add flag to force aimdo to access model files through the OS cache in Windows.

[brendanhoar]

This is primarly addressed to @rattus128:

Is aimdo using directio access to the model files in windows, bypassing the OS cache?

If so, could support be added to use normal IO, still with large read requests, that would go though the OS cache and allows Windows to cache the files in OS-managed RAM? This would be particularly helpful for large RAM systems that also run complex workflows with large base/refiner/video/videoswap/lora/upscaler models.

Presumably a cli flag would also be added to comfyui to pass that on to the aimdo library, so as to continue supporting the target features that you are primarily concerned about.

B

[brendanhoar]

Looks like the issue I am seeing in the OS cache seeming to be unused is more subtle.

The OS cache is "temporarily" used when the CreateFileW flag FLAG_FILE_SEQUENTIAL_ACCESS is passed since all reads must pass through the cache in modern Windows. However, the flag tells the OS cache to mark cached data behind the current file pointer (e.g. up to the point the application has read the data from the OS) as the highest priority cache data to dispose of, which then gets freed even without OS-side RAM pressure.

Without this flag, all the file data is in the normal priority disposal state and until RAM pressure hits, it usually sticks around in RAM, only being disposed of in a least most-recently read order under pressure.

While the flag does double the readahead amount, the OS infers that the calling application is entirely responsible for caching file data and so it also rapidly discards cache data already handed off to the app when this flag is used.

The end result is that whenever aimdo purges data it owns for whatever reason, there's never much of a chance that data will still be held in the OS cache. Therefore aimdo ends up forcing storage hardware rereads whenever it needs data aimdo doesn't already hold in memory, because what the OS usually would have cached with default CreateFileW flag conditions is now long gone when FLAG_FILE_SEQUENTIAL_ACCESS was used.