Outgoing client Certificate message is not fragmented according to negotiated Max Fragment Length

[pbizzarriINIM]

I am testing an embedded mutual-TLS client based on Mbed TLS 3.6.6 with the following options enabled:

#define MBEDTLS_SSL_VARIABLE_BUFFER_LENGTH
#define MBEDTLS_SSL_MAX_FRAGMENT_LENGTH

The device uses TLS 1.2 mutual authentication. The current certificate chains are relatively small:

• server Certificate handshake message: about 1200 bytes
• client Certificate handshake message: about 900 bytes

With:

#define MBEDTLS_SSL_IN_CONTENT_LEN   4352
#define MBEDTLS_SSL_OUT_CONTENT_LEN  4352

the handshake succeeds.

I then tested TLS Maximum Fragment Length negotiation.

When using MBEDTLS_SSL_MAX_FRAG_LEN_512, the peer accepts the MFL extension. The server-side Certificate message is received correctly by the device, fragmented across multiple TLS records within the negotiated 512-byte limit.

However, when Mbed TLS sends the client Certificate message, it appears to emit it as a single TLS record of about 833 bytes. The peer then aborts the handshake with a fatal record_overflow alert.

With MBEDTLS_SSL_MAX_FRAG_LEN_4096, the same handshake succeeds, because both the server and client Certificate messages fit below the negotiated 4096-byte record limit.

This looks like an asymmetry in MFL handling:

• incoming Certificate messages are handled correctly when the peer fragments them according to MFL;
• outgoing client Certificate messages do not appear to be fragmented by Mbed TLS according to the negotiated MFL.

Expected behavior:

After a Maximum Fragment Length value has been negotiated, Mbed TLS should ensure that all outgoing TLSPlaintext records, including handshake records such as the client Certificate message, respect the negotiated maximum fragment length.

Actual behavior:

The outgoing client Certificate message appears to be sent as a single TLS record larger than the negotiated MFL value, causing the peer to abort with record_overflow.

This is relevant for embedded mutual-TLS devices, where MFL is used to reduce runtime RAM usage while still supporting certificate-based authentication.

Please let me know if this is a known limitation, a configuration issue, or an actual bug in the outgoing handshake fragmentation path.

[pbizzarriINIM]

ISSUE_WIRESHARK_log.zip

I am attaching a Wireshark capture that reproduces the issue with MBEDTLS_SSL_MAX_FRAG_LEN_512.

Notes about the capture:

• the PKI used in this test is a temporary lab PKI generated with OpenSSL, not a production PKI;
• the visible cloud address is a public lab endpoint;
• the device address is an internal LAN address;
• no private keys are included.

What the capture shows:

• the client sends max_fragment_length = 512 in ClientHello;
• the server accepts it and replies with max_fragment_length = 512 in ServerHello;
• the server Certificate message, about 1200 bytes, is fragmented correctly into TLS records within the 512-byte limit;
• the Mbed TLS client Certificate message, about 829 bytes, is sent as a single TLS handshake record of about 833 bytes;
• the server then aborts with a fatal record_overflow alert.

This is the relevant device-side log excerpt from the same test. The timestamp should be considered relative time only:

00:00:11,219 Socket SW/CLOUD [IF id] - [1 4] CONNECTED
00:00:11,220 TLS HS queued slot=0 class=1
00:00:11,221 MRM heap [prepare_global_configs_ok] cur=3352B/34blk peak=4840B/45blk dpk=1488B heap=19456
00:00:11,222 TLS HS start slot=0 class=1 hard=8000 idle=2500
00:00:11,245 Socket [IF id] [1 4] Service [1 0] - RX DATA
00:00:11,248 MRM TLS HS large calloc kept on BASE: req=1203 slot=0 state=3
00:00:11,249 MRM TLS HS large calloc kept on BASE: req=677 slot=0 state=3
00:00:11,257 MRM TLS HS large calloc kept on BASE: req=513 slot=0 state=3
00:00:11,264 MRM TLS HS large calloc kept on BASE: req=588 slot=0 state=3
00:00:11,441 MRM TLS HS large calloc kept on BASE: req=833 slot=0 state=8
00:00:11,457 MRM TLS HS large calloc kept on BASE: req=2043 slot=0 state=8
00:00:11,529 Socket [IF id] [1 4] Service [1 0] - RX DATA
00:00:11,546 runtime socket [4] error
00:00:11,547 Connection error
00:00:11,547 MRM TLS HS large calloc kept on BASE: req=2344 slot=0 state=9
00:00:11,551 MRM TLS HS large calloc kept on BASE: req=2344 slot=0 state=9
00:00:11,594 TLS HS failed slot=0 ret=-27648
00:00:11,596 Closed gen socket - slot [0] IF [1] id [4]

I can also provide a run with Mbed TLS debug logs enabled if that would help the investigation.

[pbizzarriINIM]

I am also attaching a cleaned Mbed TLS verbose log from another test performed with mbedtls_debug_set_threshold(4);

The log shows that max_fragment_length is negotiated, and that the incoming server Certificate message is correctly received as fragmented records and reassembled.

However, when the client sends its own Certificate message, Mbed TLS writes it as a single handshake record with msglen = 833, instead of fragmenting it according to the negotiated 512-byte MFL.

The log confirms that incoming handshake fragmentation/reassembly works correctly: the server Certificate message (hslen = 1203) is received as multiple fragments (512 + 512 + 179) and then reassembled.
The issue is on the outgoing side: when writing the client Certificate message, Mbed TLS emits a single TLS handshake record with msglen = 833, after max_fragment_length = 512 has been negotiated.

The later -0x6c00 failure happens after this oversized client Certificate record has already been sent, due TLS fatal record_overflow alert from the server.

log_mbedtls_verbose_clean_redacted.zip

[pbizzarriINIM]

I tested a small diagnostic patch that only changes the TLS stream outgoing handshake record framing path.

With the same client certificate and max_fragment_length = 512, the client Certificate message is no longer emitted as a single msglen = 833 TLS record, but is split into multiple TLS records within the negotiated MFL.

The patch does not change certificate construction, buffering policy, MBEDTLS_SSL_VARIABLE_BUFFER_LENGTH, allocators, or post-handshake buffer resizing. It only demonstrates that the issue is localized in the TLS stream outgoing handshake record fragmentation path.

This is only a reference/diagnostic patch, not an upstream-ready fix, because a complete implementation should also preserve fragmentation progress correctly when a non-blocking BIO returns WANT_WRITE in the middle of a multi-record handshake message.

ssl_msg.c.patched_beta_mfl_outfrag.c
ws_patch_HS.zip