ModelPruning callback - hard pruning

[ilya-SX]

Hi all,

I am performing a structural pruning using the pruning callback (pytorch_lightning.callbacks.ModelPruning) and since it performs soft pruning only (replacing weights with zeros) I see almost no difference in model latency when converted to ONNX. It seems that to reduce model latency hard pruning (removing zero weights) should be performed.

My questions are:

1. Is it possible to perform hard pruning using this callback?
2. What is the goal of soft pruning (besides slight latency reduction and dropout-like effect)? What is the point of using this callback if it doesn't actually reduce latency (the goal of pruning)?

Thanks