ESP32-S3-zero I2C Slave issues

[Pepito-design]

I'm trying to get a pair of ESP32-S3-zero connected via I2C.
Arduino IDE, on Linux, all up to date.
One ESP32-S3-zero is I2C Master for AD-converter, works perfect.
The same device is acting as I2C Slave for the other ESP32-S3-zero board ( I2C Master ).
The Master-Slave communication starts very well, Master is able to receive 4 bytes from Slave.
But after a some minutes with 5 transfers/sec,
the first received byte is no longer the expected data byte, it's the ( slave address shifted left by 1 ) OR 0x01.
The expected data byte is moved to the next position, 2. -> 3., 3. -> 4. and the 4. is no longer there.
I found out, I'm not alone with that issue.
But I can't find a solution.
I tried multiple ESP32-S3 modules, always the same.
Is it a known hardware problem?
Pull-UP's are 1,5k, SDA and SCL are looking perfect. Is independent of SCL frequency and I2C channel,
exchange twowire(0) and twowire(1) did not help.

Any suggestion? Tektronix THS730A is available, wires are short and soldered.
So it's not a "typical bread board" problem.