feat: implementing a resource manager to prevent resource exhaustion attacks - #836
Conversation
Sahilgill24
changed the title
|
@Sahilgill24 : Hi Sahil. Thank you for submitting the PR. Appreciate it. Please resolve the CI/CD issues. CCing @acul71 and @sumanjeet0012, who will help you arrive at a good conclusion on the issue. |
The CI/CD issues could be resolved, but commits are just the basic structure of the project, so let's here form @Sahilgill24 when he planned to work on this. |
|
@acul71 : Sure, thank you for your feedback. Appreciate it. As recommended by you in the recent maintainer call, wish to request @Sahilgill24 to add @acul71 (Luca) and @bomanaps (Mercy) as collaborators to this project. They wish to contribute alongside on this important issue raised by rust-libp2p team. |
|
@acul71 : Please let us know if there are any bottlenecks faced in this PR. CCing @lla-dane and @sumanjeet0012. We have been advised to have this PR landed soon. |
@Sahilgill24 don't respond, and I don't think we have been added as collaborators. |
can you add me and @bomanaps as collaborator (to be able to push to your repo/branch) |
|
@seetadev Working along this with @Sahilgill24 |
- Updated allowlist.py to use Optional for config parameter. - Introduced Direction enum in limits.py for connection direction clarity. - Enhanced metrics.py with type hints for data structure. - Improved scope.py with type annotations and method signatures. - Removed empty manager.py and manager_old.py files for cleanup.
|
@bomanaps @Sahilgill24 |
|
Thanks alot for the detailed analysis @acul71 . Also I would like you, @yashksaini-coder , @seetadev to try out the metrics dashboard based on these instructions on your devices once. Instructions
on the second window, run the example now open localhost:3000 on your browser, This should ideally open the grafana dashboard , open the dashboards tab, now open the py-libp2p resource manager and you will be able to see the 8 dashboards showing the resource consumption stats and for this particular example you would be able to see the graphs |
… swarm and host classes - Added optional resource manager parameter to new_swarm and new_host functions. - Updated BasicHost and RoutedHost classes to accept and utilize ResourceManager. - Configured Sphinx documentation to clarify ambiguous references related to ResourceManager. - Implemented resource scope management in SwarmConn for improved connection handling.
…m functions; update related tests for consistency
…ports - Added resource manager support to TCPListener and WebsocketListener for connection limits. - Integrated resource availability checks in TCP and WebSocket transport classes. - Enhanced initialization of TCPListener and WebsocketListener to accept resource manager and max connections parameters. - Updated related classes to propagate resource manager settings to transport layers.
in TCP and WebSocket transports" This reverts commit deb0c95.
- Introduced automatic resource manager creation in new_host and new_swarm functions for improved connection and stream protection. - Implemented pre-upgrade admission checks in Swarm for outbound connections using the resource manager. - Added support for scoped stream limits based on service and protocol in ResourceManager. - Enhanced connection rate limiting and CIDR limits for better resource control. - Updated documentation to reflect new resource management features and usage examples.
- Introduced CIDRRule and CIDRLimiter classes to manage connection limits based on CIDR networks. - Implemented methods to allow, acquire, and release connections while enforcing maximum connection limits per subnet. - Enhanced thread safety with RLock for concurrent access to connection counts.
- Introduced tests for hierarchical scopes, pre-upgrade admission, and rate limiting in ResourceManager. - Updated existing tests to import Direction from the correct module. - Enhanced coverage for connection limits based on service and protocol.
- Deleted Prometheus and Grafana configuration files, along with example monitoring scripts, to streamline the resource manager's functionality. - This cleanup removes unused monitoring components that are no longer needed.
- Set enable_connection_pooling and enable_memory_pooling to False to prevent memory spikes during tests. - Adjusted connection pool initialization to avoid eager pre-allocation and set initial_blocks to 0 in memory pool to reduce upfront allocations.
- Enhanced readability of argument parsing in configure.py and run_demo.py by formatting multi-line arguments. - Updated regex pattern in set_exporter_port function for clarity. - Improved logging statements for connection and stream management to enhance debugging information.
Add more arguments to have more extensive use-case scenarios for monitoring services test and demonstration
|
@acul71 @pacrob @seetadev Hi, I've noticed that in some py-cid checks, the workflows runs is either gets stucked on core tests, or unexpectedly failed with error code 143. Also I also experienced heavy lag, and computer freezes during local testing. This happened after completing the test additions, resolving gaps, and completing the recommendations and all the other optional improvements. Requesting a code review on new updated changes |
|
@yashksaini-coder : Thanks for sharing. This will be fixed this week via self hosted runners. |
…ests - Added handling for cancelled tasks in ProviderStore to log timeouts. - Updated the test for provider discovery to include retry logic for improved robustness against race conditions, increasing sleep duration for propagation.
This new asynchronous function waits for a peer record to become available in the peerstore, addressing timing discrepancies across platforms. The test_find_node function has been updated to utilize this utility, ensuring peer records are retrieved reliably even when immediate availability cannot be guaranteed.
|
@yashksaini-coder and @Sahilgill24: Hi Yash and Sahil. Appreciate your efforts. Wish to share that we will have the advancements in metrics module tackled in a new PR. I have studied the js-libp2p & go-libp2p metrics module and there are certain todos, which will be work on a new PR. I will collaborate with you on these efforts soon. Doing a final review and merging this PR. |
|
@Sahilgill24 @yashksaini-coder While working on another feature, I noticed a possible inconsistency. Could you please confirm if this is an error or if I might be misunderstanding it? The Warning Encountered
|
|
@sumanjeet0012 : Thanks for sharing, Sumanjeet. Appreciate it. Indeed, this is not as an expected behavior. Please open an issue and assign it to me @yashksaini-coder and @Sahilgill24. CC me too. |
6 participants
What was wrong?
Issue #572
How was it fixed?
This is currently the implementation of the MVP resource manager, more details about the implementation plan are added in the #572 itself.
1.) The main components of the resource manager are the scopes and the limits, so we will begin by just implementing them and then move on to others like metrics and garbage collection. The below diagram shows the folder structure
2.) A base ResourceManager class would be our main data structure similiar to the go-libp2p's struct and would be defined in the manager.py file and also include the logic for resource transfer from different scopes to their children scopes.
3.) Each scope would be a different Class as well in the scope.py file mimicking the structure of the go-libp2p's implementation and respectively writing the main functions corresponding to them.
4.) The limits for each scope would also be defined in the limits.py file along with the limiter interface. The allowlist.py file would manage the allowlists for all the scopes as well as contain the default parameters.
To-Do