Skip to content

Firewall: keep the TCP hole open for the final ACK - #111

Open
nding0405 wants to merge 3 commits into
CHERIoT-Platform:mainfrom
nding0405:firewall-block-last-ack
Open

Firewall: keep the TCP hole open for the final ACK#111
nding0405 wants to merge 3 commits into
CHERIoT-Platform:mainfrom
nding0405:firewall-block-last-ack

Conversation

@nding0405

Copy link
Copy Markdown
Contributor

During an active TCP close, FreeRTOS calls on_tcp_connect before sending the final ACK. At that point, the callback removed the TCP firewall hole, so the firewall dropped the ACK and the peer could not finish closing. Waiting for the TCP state change is not enough because the state changes before the ACK passes through egress.

Keep the firewall hole open until the final ACK passes through egress. on_tcp_connect marks the hole as closing, egress marks it as safe to remove, and network_socket_close() removes it afterward.

Key changes:

  • Add a state field to each TCP firewall hole: InUse for an open connection, InTermination while waiting for the final ACK, and CanBeRemoved after that ACK passes egress. NotFound means that no matching hole exists.
  • Change the SmallTableBase helpers to compare only each table's key fields. The endpoint key remains the address and ports, so existing helper calls keep their old behavior and ignore the new state field. Lookup remains O(log n); only the close helpers inspect the state.
  • Change the hole from InUse to InTermination in on_tcp_connect.
  • Change it from InTermination to CanBeRemoved after the next matching packet passes egress.
  • Make network_socket_close() wait for CanBeRemoved before removing the hole. A successful return guarantees that the hole was removed.

Note:
The polling here can be removed for ipv4 path, but I keep it here for ipv6. Since the firewall endpoint delayed removal and polling for firewall state logics are currently only applied to ipv4 path. The polling here still make sure we are at a safe point to free the socket, but it is not a signal of we can safely remove the firewall endpoint.

During an active TCP close, FreeRTOS calls `on_tcp_connect` before
sending the final ACK. At that point, the callback removed the TCP
firewall hole, so the firewall dropped the ACK and the peer could not
finish closing. Waiting for the TCP state change is not enough because
the state changes before the ACK passes through egress.

Keep the firewall hole open until the final ACK passes through egress.
`on_tcp_connect` marks the hole as closing, egress marks it as safe to
remove, and `network_socket_close()` removes it afterward.

Key changes:

- Add a state field to each TCP firewall hole: `InUse` for an open
  connection, `InTermination` while waiting for the final ACK, and
  `CanBeRemoved` after that ACK passes egress. `NotFound` means that no
  matching hole exists.
- Change the `SmallTableBase` helpers to compare only each table's key
  fields. The endpoint key remains the address and ports, so existing
  helper calls keep their old behavior and ignore the new state field.
  Lookup remains `O(log n)`; only the close helpers inspect the state.
- Change the hole from `InUse` to `InTermination` in `on_tcp_connect`.
- Change it from `InTermination` to `CanBeRemoved` after the next
  matching packet passes egress.
- Make `network_socket_close()` wait for `CanBeRemoved` before removing
  the hole. A successful return guarantees that the hole was removed.
Comment thread lib/firewall/firewall.hh Outdated
* - `CanBeRemoved`: Egress passed the final packet; the hole may be removed.
* - `NotFound`: No hole matches the address and ports.
*/
enum class TCPFirewallState : uint32_t

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we make the base type smaller than this (e.g. uint8_t?) so that it fits better in the small table?

Comment thread lib/firewall/firewall.hh
* Get a TCP hole's state without changing it.
*
* Returns the state, or `TCPFirewallState::NotFound` if no hole matches.
*/

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that this can also fail for normal reasons so the return value might be a negative errno value.

Change firewall_get_tcpipv4_endpoint_state() to return int so it can
return negative error codes.

Change TCPFirewallState from uint32_t to uint8_t.
@nding0405
nding0405 force-pushed the firewall-block-last-ack branch from ebf53a8 to c4c4eb5 Compare August 8, 2026 03:44
@davidchisnall

Copy link
Copy Markdown
Contributor

Why the change to remove the comments on the values? This makes IDE integration worse.

Updated comments for TCPFirewallState enum values for clarity.
@nding0405

Copy link
Copy Markdown
Contributor Author

Why the change to remove the comments on the values? This makes IDE integration worse.

Fixed. I restored the per-value comments. I had moved them out since I thought it might look a bit cleaner, but I see why per-value is better. Could you please take another look? Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants