feat: improve scanning speed

[SWvheerden]

Description

Motivation and Context

How Has This Been Tested?

What process can a PR reviewer use to test or verify this change?

Breaking Changes

• None
• Requires data directory on base node to be deleted
• Requires hard fork
• Other - Please specify

[gemini-code-assist]

Code Review

This pull request refactors the blockchain scanning logic to use an asynchronous streaming approach. The BlockchainScanner trait and its gRPC and HTTP implementations now return a tokio::sync::mpsc::Receiver instead of a single result vector, allowing for more efficient processing of scanned blocks. The HTTP scanner specifically adopts a producer-consumer model using tokio::spawn and rayon for concurrent downloading and processing. Additionally, the ScanCoordinator has been updated to handle these streaming results and includes new logic for partial scan modes. One piece of feedback is to remove a dbg! macro used for timing that was left in the coordinator code.

[SWvheerden]

/gemini review

[gemini-code-assist]

Code Review

This pull request refactors the blockchain scanning interface to use asynchronous streaming via mpsc channels for both gRPC and HTTP implementations. It introduces retry logic with exponential backoff for HTTP requests and updates the ScanCoordinator to handle the new streaming architecture. Feedback identifies a potential memory issue in gRPC height collection, a bug in progress tracking within the coordinator, and the need to move partial scan limit checks inside the processing loop. Additionally, it is recommended to remove commented-out code and ensure consistent retry logic across all scanner implementations.