[codex] Implement transport peer selection modes#7853
Conversation
|
If you're new to commit signing, there are different ways to set it up: Sign commits with
|
There was a problem hiding this comment.
Code Review
This pull request introduces new transport types (TorTcp and TcpTor) to support preferred peer selection and dial address ordering based on transport protocols. The reviewer provided valuable feedback on optimizing address filtering to avoid cloning metadata, using sort_by_cached_key to prevent redundant string allocations during peer sorting, and addressing a potential non-determinism issue in SQLite when ordering distinct queries.
| let addresses = peer | ||
| .addresses | ||
| .clone() | ||
| .into_vec() | ||
| .into_iter() | ||
| .filter(|address| { | ||
| let protocol = TransportProtocol::from(address); | ||
| !excluded_dial_addresses | ||
| .iter() | ||
| .any(|excluded| excluded.contains(address)) | ||
| && preferred_transport_protocols.contains(&protocol) | ||
| }) | ||
| .collect::<Vec<_>>(); |
There was a problem hiding this comment.
Cloning the entire MultiaddressesWithStats struct and converting it to a vector is inefficient because it clones all address statistics and metadata only to discard them. Instead, you can iterate over the addresses slice directly using .addresses() and clone only the Multiaddr values that match the filter criteria.
let addresses = peer
.addresses
.addresses()
.iter()
.filter_map(|address| {
let addr = address.address();
let protocol = TransportProtocol::from(addr);
if !excluded_dial_addresses.iter().any(|excluded| excluded.contains(addr))
&& preferred_transport_protocols.contains(&protocol)
{
Some(addr.clone())
} else {
None
}
})
.collect::<Vec<_>>();There was a problem hiding this comment.
Current patch already iterates peer.addresses.addresses() and only clones the Multiaddr values that pass the filter. It does not clone the whole MultiaddressesWithStats value.
| fn order_peers_by_node_id_order(peers: &mut [Peer], node_ids: &[String]) { | ||
| let node_id_positions = node_ids | ||
| .iter() | ||
| .enumerate() | ||
| .map(|(index, node_id)| (node_id.clone(), index)) | ||
| .collect::<HashMap<_, _>>(); | ||
|
|
||
| peers.sort_by_key(|peer| { | ||
| node_id_positions | ||
| .get(&peer.node_id.to_hex()) | ||
| .copied() | ||
| .unwrap_or(usize::MAX) | ||
| }); | ||
| } |
There was a problem hiding this comment.
Using sort_by_key with peer.node_id.to_hex() will perform a string allocation and hex conversion O(n log n) times during sorting. Using sort_by_cached_key instead ensures that the key function is evaluated exactly once per element, reducing the number of allocations to O(n).
| fn order_peers_by_node_id_order(peers: &mut [Peer], node_ids: &[String]) { | |
| let node_id_positions = node_ids | |
| .iter() | |
| .enumerate() | |
| .map(|(index, node_id)| (node_id.clone(), index)) | |
| .collect::<HashMap<_, _>>(); | |
| peers.sort_by_key(|peer| { | |
| node_id_positions | |
| .get(&peer.node_id.to_hex()) | |
| .copied() | |
| .unwrap_or(usize::MAX) | |
| }); | |
| } | |
| fn order_peers_by_node_id_order(peers: &mut [Peer], node_ids: &[String]) { | |
| let node_id_positions = node_ids | |
| .iter() | |
| .enumerate() | |
| .map(|(index, node_id)| (node_id.clone(), index)) | |
| .collect::<HashMap<_, _>>(); | |
| peers.sort_by_cached_key(|peer| { | |
| node_id_positions | |
| .get(&peer.node_id.to_hex()) | |
| .copied() | |
| .unwrap_or(usize::MAX) | |
| }); | |
| } |
There was a problem hiding this comment.
Current patch already uses sort_by_cached_key here, so peer.node_id.to_hex() is evaluated once per peer during sorting.
| fn build_addr_preference_order_sql(transport_protocols: &[TransportProtocol]) -> Option<String> { | ||
| if transport_protocols.is_empty() { | ||
| return None; | ||
| } | ||
|
|
||
| let conditions = transport_protocols | ||
| .iter() | ||
| .enumerate() | ||
| .map(|(index, protocol)| { | ||
| format!( | ||
| "WHEN multi_addresses.address LIKE '{}%' THEN {}", | ||
| sql_escape(protocol.get_prefix()), | ||
| index | ||
| ) | ||
| }) | ||
| .collect::<Vec<_>>(); | ||
|
|
||
| Some(format!("CASE {} ELSE {} END", conditions.join(" "), transport_protocols.len())) | ||
| } |
There was a problem hiding this comment.
When a peer has multiple addresses (e.g., both Onion and IPv4), the joined query produces multiple rows for that peer. Because get_available_dial_candidates uses DISTINCT peers.node_id, SQLite collapses these rows. Ordering by multi_addresses.address (via order_sql) on a DISTINCT query is non-deterministic in SQLite because the value used for ordering the collapsed row is arbitrary. This can lead to incorrect candidates being selected when LIMIT is applied.
There was a problem hiding this comment.
This query does not use DISTINCT; it selects ordered address rows, over-fetches by MAX_MULTIADDRESSES_PER_PEER, deduplicates node IDs in Rust while preserving row order, then fetches the peers and reapplies transport preference in memory. That avoids SQLite choosing an arbitrary address value for a collapsed distinct peer row.
dd9e230 to
e68f552
Compare
e68f552 to
8e94e76
Compare
| .await? | ||
| }, | ||
| TransportType::Tor => { | ||
| TransportType::Tor | TransportType::TorTcp | TransportType::TcpTor => { |
There was a problem hiding this comment.
you need the handle the cases above also
| self.connection_manager_config.transport_protocols = protocols.clone(); | ||
| self.transport_protocols = protocols; |
There was a problem hiding this comment.
why clone it and store it twice
| supported_transport_protocols: &[TransportProtocol], | ||
| preferred_transport_protocols: &[TransportProtocol], |
There was a problem hiding this comment.
these seem like duplicates?
| query = query.limit(limit); | ||
| } | ||
|
|
||
| let node_ids = Self::unique_node_ids_preserving_order(query.load::<String>(conn)?, Some(n)); |
There was a problem hiding this comment.
this combo will break the limit and qeury more
|
closing in favour of #7851 |
Fixes #7830
Summary
Tor,Tcp,TorTcp, andTcpTor, withTcpToras the new default.MemoryandSocks5transport variants for compatibility.Validation
cargo test -p tari_p2p --no-default-featurescargo test -p tari_comms transport_protocol_preference --no-default-featurescargo test -p tari_integration_tests --test cucumber --no-rungit diff --checkNotes
cargo test -p tari_comms --no-default-featuressuite locally. The transport-focused tests passed, but the full package run hit local baseline failures in temp SQLite/DNS-related tests outside this change.cargo fmt --checkis not usable in my local stable toolchain for this repo becauserustfmt.tomlenables nightly-only rustfmt options and stable reports broad baseline diffs. No source files were modified by that check.