You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Implement tofile on tensors to reduce data write time by 40% (#210)
This PR introduces the `tofile` method on tensors (similarly named as
the one on numpy arrays), which allows for faster write and lower memory
usage on external data by bypassing tobytes().
Compatibility with existing `TensorProtocol`s is maintained in the
external data module by using `tofile` only when it is available in the
class. The `TorchTensor` class in PyTorch exporter should be updated
accordingly to leverage the new logic when saving.
Note that io time to disk is reduced by 40% below.
> [!NOTE]
> TensorProtocol is not updated because we do isinstance() checks on
external implementations (PyTorch). Adding the method in the protocol
will cause isinstance check to fail on those implementations that have
not added the tofile method.
Reference:
https://github.com/microsoft/onnxscript/pull/2241/files/b2381658492510a9bcc8c0a8574db7368e33bceb
Before:
```
________________________________________________________
Executed in 48.08 secs fish external
usr time 60.54 secs 0.00 millis 60.54 secs
sys time 23.06 secs 1.22 millis 23.06 secs
```
<img width="2325" height="1136" alt="image"
src="https://github.com/user-attachments/assets/5283057d-c401-41f1-98f0-e11aa3707591"
/>
<img width="1225" height="1236" alt="image"
src="https://github.com/user-attachments/assets/5d900bd1-4283-4332-9ec3-dd2bd30f8ae3"
/>
After:
```
________________________________________________________
Executed in 45.69 secs fish external
usr time 60.68 secs 244.00 micros 60.68 secs
sys time 22.22 secs 518.00 micros 22.22 secs
```
<img width="2332" height="1247" alt="image"
src="https://github.com/user-attachments/assets/87daff64-5b39-4d54-a9c8-98e7ec339033"
/>
<img width="1239" height="1236" alt="image"
src="https://github.com/user-attachments/assets/d1b87f94-f463-4e45-afcc-577f9a5f7c91"
/>
Fix#207
---------
Signed-off-by: Justin Chu <[email protected]>
0 commit comments