pooling2d_cl (io_stream) line buffer synthesized with depth 1 in Vitis HLS 2025.1 → C/RTL co-simulation mismatch for MaxPooling2D 2×2 #1592

Quick summary

With io_stream, a MaxPooling2D(pool_size=2) layer gives correct results in C simulation but wrong results in RTL co-simulation. In RTL, the line buffer ap_shift_reg<T, in_width> line_buffer[MAX(pool_height-1, 1)][n_filt] in nnet::pooling2d_cl (nnet_pooling_stream.h) behaves as if its depth were 1 instead of in_width. Each output therefore takes the max over the current row and the previous pixel, not the previous image row.

The trigger appears to be the first array dimension being 1 (pool_height - 1 = 1 for 2×2 pooling). Declaring the array with first dimension 2 (only index 0 is used) gives bit-exact csim = RTL. Conv2D 3×3 layers use the same nnet::shift_line_buffer / ap_shift_reg with a [2][n_chan] array and are correct in RTL.

hls4ml version

hls4ml 1.3.0 (Python 3.10.12, TensorFlow 2.13.0, QKeras 0.9.0; Ubuntu 22.04 on WSL2)

Frontend

Keras v2

Backend

Vitis

io_type

io_stream

HLS tool and version

Vitis HLS 2025.1 (Build 6135595), Windows 11, run via vitis-run --mode hls --tcl build_prj.tcl

Steps to reproduce

Minimal reproducer: MaxPooling2D (io_stream) C/RTL co-simulation mismatch
with hls4ml 1.3.0 + Vitis HLS 2025.1.

1) python repro_pool_cosim.py            -> writes ./pool_repro (hls4ml project)
2) cd pool_repro && vitis-run --mode hls --tcl build_prj.tcl
   (build_opt.tcl is set to csim + synth + cosim)
3) python repro_pool_cosim.py compare    -> compares csim vs RTL output

Optional: python repro_pool_cosim.py patch   -> applies the workaround
(line_buffer first dimension 1 -> 2) to ./pool_repro before step 2.

import os
import re
import sys

import numpy as np

OUT = 'pool_repro'
SOL = os.path.join(OUT, 'myproject_prj', 'solution1')


def build():
    import hls4ml
    from tensorflow.keras import layers, models

    inp = layers.Input((16, 16, 1), name='input_1')
    out = layers.MaxPooling2D(pool_size=2, name='pool')(inp)
    model = models.Model(inp, out)

    rng = np.random.default_rng(0)
    x = np.round(rng.uniform(0, 4, (1, 16, 16, 1)) * 64) / 64  # exactly representable in fixed<16,6>
    y = model.predict(x, verbose=0)
    np.save('tb_in.npy', x.reshape(1, -1))
    np.save('tb_out.npy', y.reshape(1, -1))

    cfg = hls4ml.utils.config_from_keras_model(model, granularity='name', default_precision='fixed<16,6>')
    hm = hls4ml.converters.convert_from_keras_model(
        model, hls_config=cfg, output_dir=OUT, backend='Vitis', io_type='io_stream',
        part='xczu7ev-ffvc1156-2-e', clock_period=5,
        input_data_tb='tb_in.npy', output_data_tb='tb_out.npy')
    hm.write()
    with open(os.path.join(OUT, 'build_opt.tcl'), 'w') as f:
        f.write('array set opt {\n    reset 0\n    csim 1\n    synth 1\n    cosim 1\n    validation 0\n'
                '    export 0\n    vsynth 0\n    fifo_opt 0\n}\n')
    print('project written to', OUT)


def patch():
    h = os.path.join(OUT, 'firmware', 'nnet_utils', 'nnet_pooling_stream.h')
    src = open(h).read()
    new, n = re.subn(r'(void pooling2d_cl\(hls::stream<data_T> &data, hls::stream<res_T> &res\) \{.*?'
                     r'line_buffer\[)MAX\(CONFIG_T::pool_height - 1, 1\)\]',
                     r'\1MAX(CONFIG_T::pool_height - 1, 2)]', src, count=1, flags=re.S)
    assert n == 1, 'pattern not found'
    open(h, 'w').write(new)
    print('patched', h)


def compare():
    csim = np.loadtxt(os.path.join(SOL, 'csim', 'build', 'tb_data', 'csim_results.log')).reshape(8, 8)
    rtl = np.loadtxt(os.path.join(SOL, 'sim', 'wrapc_pc', 'tb_data', 'rtl_cosim_results.log')).reshape(8, 8)
    x = np.load('tb_in.npy').reshape(16, 16)
    ref = x.reshape(8, 2, 8, 2).max(axis=(1, 3))
    print('csim == Keras/NumPy max-pool :', np.allclose(csim, ref))
    print('RTL  == csim                 :', np.allclose(rtl, csim),
          f'({int((~np.isclose(rtl, csim)).sum())}/64 values differ)')


if __name__ == '__main__':
    {'build': build, 'patch': patch, 'compare': compare}[sys.argv[1] if len(sys.argv) > 1 else 'build']()

Expected behaviour

the RTL output (solution1/sim/wrapc_pc/tb_data/rtl_cosim_results.log) equals the csim output.

Actual behaviour

csim matches Keras/NumPy exactly, but 40/64 RTL values differ with the attached script. A 16-channel variant gave 604/1024. With the workaround below, the same script gives 0/64.

Possible fix or additional context

Analysis

• A Python re-implementation of shift_line_buffer + compute_pool_buffer_2d with line-buffer depth D = 1 reproduces the RTL output exactly (64/64, and 1024/1024 for 16 channels). With D = in_width = 16 it reproduces csim exactly.
• The failure does not depend on hls4ml configuration: it reproduces with config_from_keras_model defaults, with StreamDepth = 1024, with 1 or 16 channels, and inside a larger U-Net.
• Changing the pragmas (INLINE OFF on compute_pool_buffer_2d, removing INLINE recursive from pooling2d_cl) does not change the RTL output.
• A Conv2D 3×3 with 16 input channels (same shift_line_buffer, line buffer [2][n_chan]) is bit-exact in RTL.
• ConvImplementation='Encoded' is not an alternative for pooling with the Vitis backend: csim aborts on the template's assert in pooling2d_cl (Only "linebuffer" implementation is supported in Vitis HLS.).

Workaround

In our tests it gives bit-exact csim = RTL:

• on the single-layer reproducer;
• on the U-Net's first pooling layer in isolation;
• on the full U-Net (2 MaxPooling layers, 64×64 output): unpatched, csim vs RTL MAE ≈ 0.12; patched, the 4096 outputs are bit-identical.

--- a/hls4ml/templates/vivado/nnet_utils/nnet_pooling_stream.h
+++ b/hls4ml/templates/vivado/nnet_utils/nnet_pooling_stream.h
@@ void pooling2d_cl(hls::stream<data_T> &data, hls::stream<res_T> &res) {
-    static ap_shift_reg<typename data_T::value_type, CONFIG_T::in_width> line_buffer[MAX(CONFIG_T::pool_height - 1, 1)]
+    static ap_shift_reg<typename data_T::value_type, CONFIG_T::in_width> line_buffer[MAX(CONFIG_T::pool_height - 1, 2)]
                                                                                     [CONFIG_T::n_filt];

Cost on a 16×16×16 pooling layer: +832 FF, +512 LUT, same latency. On the full U-Net (pooling layers 16×16×16 and 8×8×32): +2,496 FF and +1,536 LUT in total (+1.5 % / +0.45 %), same latency. The original design is only "cheaper" because it implements a 1-element buffer.

An alternative that also works but costs more (+6.3k FF): replace ap_shift_reg with an explicit, fully partitioned array shift register.

The same pattern (MAX(..._height - 1, 1) with a 2-row kernel) may affect other layers using ap_shift_reg line buffers with a first dimension of 1. We have only verified 2D pooling.

Additional notes

• We found this while validating a U-Net for electrical impedance tomography on a ZCU104. At the network output the mismatch was MAE ≈ 0.12 (on a 0–1 scale) between csim and RTL.
• With Vitis HLS 2025.1, build_prj.tcl also prints ERROR: [HLS 200-101] config_array_partition: Unknown option '-maximum_size' (non-fatal). This may be worth a separate check.
• Separate flow issue (possibly Windows-only): the validation step in build_prj.tcl always reports Test failed, even when csim and RTL are identical. The testbench writes tb_data/*_results.log relative to its working directory (solution1/csim/build/ and solution1/sim/wrapc_pc/, where Vitis copies tb_data). The validation compares ./tb_data/*.log in the project root, which never exist, and compare_files returns 0 for missing files. Copying both logs to ./tb_data/ after csim and after cosim (and deleting stale copies at the start) makes the check report PASSED or FAILED correctly in our tests. Happy to open a separate issue if preferred.

AI assistance

• An AI tool helped produce this report, and I have checked its content myself.