Parallelize object purge #3
Description
When deleting objects from a store during a purge, deletion is done synchronously.
To speed up process, this could be parallelized. Depending on the backend:
- Mnesia: simply spawning tasks?
- Riak: the connection pool will quickly be exhausted. To be assessed:
- Task.Supervisor.async_stream (but how to determine the number of parallel processes?)
- GenStage
Also, the purge process is synchronous. If the purge interval is 10 minutes, and a purge process lasts 2 minutes, the next pruge process will be triggered in 12 minutes. It prevents having several purge processes launched in extreme cases (when the purge process takes longer than the purge interval), but should it be so?