[rcore][drm] SUPPORT_DRM_CACHE stops presenting after 2 page flips on Mali (RK3326)

[jtakakura]

Please, before submitting a new issue verify and check:

• I tested it on latest raylib version from master branch
• I checked there is no similar issue already reported
• I checked the documentation on the wiki
• My code has no errors or misuse of raylib

Issue description

Follow-up to #4988, where more testing and benchmarking was asked for before making the cached path the default. I have an R36S, which uses ARM's Mali blob rather than mesa, so the results are different from the Raspberry Pi it was developed on.

With SUPPORT_DRM_CACHE defined, the screen never updates. Not "runs and then freezes" — nothing appears after the first couple of frames. The game loop is unaffected: my log shows it running at 39-60 fps, button presses still change scenes, and the program exits cleanly. Only presentation is dead.

I interposed on libdrm and gbm to see what happens:

addfb      call=1 rc=0 handle=1
addfb      call=2 rc=0 handle=2
addfb      call=3 rc=0 handle=4
lock_front call=1 bo=0x512a790 NEW distinct=1
lock_front call=2 bo=0x52390d0 NEW distinct=2
lock_front call=3 bo=0x5238b20 NEW distinct=3
lock_front call=4 bo=0x51ab430 NEW distinct=4
pageflip   call=1 rc=0
pageflip   call=2 rc=0

After this there are no further drmModeAddFB and no further drmModePageFlip calls for the rest of the session, while gbm_surface_lock_front_buffer() and gbm_surface_release_buffer() keep being called every frame. lock_front never returns NULL, so this is not buffer exhaustion.

gbm_surface_lock_front_buffer() hands out a 4th distinct gbm_bo pointer on this driver. fbCache holds MAX_DRM_CACHED_BUFFERS = 3 and GetOrCreateFbForBo() has no eviction, so from the 4th buffer on it takes

if (fbCacheCount >= MAX_DRM_CACHED_BUFFERS) return 0; // FB cache full

and SwapScreenBuffer() releases the buffer and returns before the page flip, every frame, permanently. errCnt[2] counts exactly this, but the block that prints the counters is commented out.

I would guess mesa reuses the same three buffers, which would explain why this does not show up on a Pi.

Would keying off gbm_bo_get_user_data() instead of the fixed array work? The code already calls gbm_bo_set_user_data() with the fb id for DestroyFrameBufferCallback.

The default path works fine on the same device: 25.57 ms per frame (39.1 fps, exactly two vblanks of the 78.2 Hz panel), and it stays at 25.57 ms even with 24 full-screen alpha-blended quads. So the cached path would be a large win here if it worked.

I am happy to test patches, I have the hardware.

Environment

• Device: R36S (Rockchip RK3326, Cortex-A35), ArkOS (Ubuntu 19.10, glibc 2.30)
• GPU: Mali-G31, libmali-bifrost-g31-rxp0-gbm.so, OpenGL ES 3.2, EGL 1.4
• Panel: 640x480 at 78.2 Hz (DRM mode 640x480p78)
• raylib 6.0 with PLATFORM_DRM, GRAPHICS_API_OPENGL_ES2, cross-compiled with Zig. GetOrCreateFbForBo() and SwapScreenBuffer() are byte-identical between 6.0 and master (d46a593), so this is not fixed on master.

To reach this bug at all on a Mali blob you first have to get past #5777 — this driver does not export eglGetPlatformDisplay, so the binary will not start. I worked around it by defining that symbol myself and falling back to eglGetProcAddress("eglGetPlatformDisplayEXT") and then eglGetDisplay(). Separate bug, same device; mentioning it only so this is reproducible.

Issue Screenshot

Not useful here — the symptom is that the screen never changes.

Code Example

Any program is enough; it fails before anything is drawn. On a Mali device you also need the #5777 workaround described above, or it will not start.

#include "raylib.h"

int main(void)
{
    InitWindow(640, 480, "drm cache test");
    SetTargetFPS(60);

    while (!WindowShouldClose())
    {
        BeginDrawing();
        ClearBackground(GetTime() - (int)GetTime() < 0.5 ? RED : BLUE);
        EndDrawing();
    }

    CloseWindow();
    return 0;
}

Built with SUPPORT_DRM_CACHE the screen stays on one colour forever; without it, it alternates as expected.

[raysan5]

@rob-bits Please, could you take a look to this issue?

[rob-bits]

Hey, I need some time to examine the issue. I quickly took a look to my application and I see this code was used:

//init
SetConfigFlags(FLAG_VSYNC_HINT);
InitWindow(gScreenWidth, gScreenHeight, "raylib game template");
SetTargetFPS(60);
DisableEventWaiting();

//..do something here
//main loop
while(1) {
//... do somehting here
BeginDrawing();
//... do something here
EndDrawing();

SwapScreenBuffer();
PollInputEvents();
}

And in the raylib config, I enabled/disabled the following parts:

// Setting a higher resolution can improve the accuracy of time-out intervals in wait functions.
// However, it can also reduce overall system performance, because the thread scheduler switches tasks more often.
//#define SUPPORT_WINMM_HIGHRES_TIMER     1
// Use busy wait loop for timing sync, if not defined, a high-resolution timer is set up and used
#define SUPPORT_BUSY_WAIT_LOOP          1
// Use a partial-busy wait loop, in this case frame sleeps for most of the time, but then runs a busy loop at the end for accuracy
//#define SUPPORT_PARTIALBUSY_WAIT_LOOP    1
// Allow automatic screen capture of current screen pressing F12, defined in KeyCallback()
//#define SUPPORT_SCREEN_CAPTURE          1
// Allow automatic gif recording of current screen pressing CTRL+F12, defined in KeyCallback()
//#define SUPPORT_GIF_RECORDING           1
// Support custom frame control, only for advanced users
// By default EndDrawing() does this job: draws everything + SwapScreenBuffer() + manage frame timing + PollInputEvents()
// Enabling this flag allows manual control of the frame processes, use at your own risk
#define SUPPORT_CUSTOM_FRAME_CONTROL    1

@jtakakura Could you please take a look if any of these helps?

[jtakakura]

@rob-bits Thank you for the suggestions. I tested exactly that configuration on the
R36S: FLAG_VSYNC_HINT before InitWindow(), DisableEventWaiting(), and
SUPPORT_CUSTOM_FRAME_CONTROL 1 with SwapScreenBuffer() and
PollInputEvents() called from the main loop.

The result was the same. The screen never updated, and the trace showed the
same pattern: three drmModeAddFB calls, a 4th distinct buffer from
gbm_surface_lock_front_buffer(), two page flips, and then nothing.

While preparing that test I also noticed that FLAG_VSYNC_HINT seems to have
no effect on PLATFORM_DRM. As far as I can see, only the GLFW and SDL backends
handle this flag, and the eglSwapInterval() call in rcore_drm.c is
commented out. And because SUPPORT_CUSTOM_FRAME_CONTROL moves the
SetTargetFPS timing to the application, my loop ran without any frame limit,
with the same freeze. So I believe frame pacing is not related to this problem.

After that, I tried the gbm_bo_get_user_data() approach from my first
comment, and on this device it fixes the problem. I changed only the two
functions, so that the fb id is stored on the buffer object itself instead of
the fixed array:

// Destroy cached framebuffer callback, set by gbm_bo_set_user_data()
static void DestroyFrameBufferCallback(struct gbm_bo *bo, void *data)
{
    (void)bo;
    uint32_t fbId = (uintptr_t)data;

    if (fbId != 0)
    {
        TRACELOG(LOG_INFO, "DISPLAY: DRM: Framebuffer removed [%u]", (uintptr_t)fbId);
        drmModeRmFB(platform.fd, fbId); // Release DRM FB
    }
}

// Create or retrieve cached DRM FB for BO
static uint32_t GetOrCreateFbForBo(struct gbm_bo *bo)
{
    // The fb id travels with the buffer object
    uint32_t cachedId = (uint32_t)(uintptr_t)gbm_bo_get_user_data(bo);
    if (cachedId != 0) return cachedId;

    uint32_t handle = gbm_bo_get_handle(bo).u32;
    uint32_t stride = gbm_bo_get_stride(bo);
    uint32_t width = gbm_bo_get_width(bo);
    uint32_t height = gbm_bo_get_height(bo);

    uint32_t fbId = 0;
    if (drmModeAddFB(platform.fd, width, height, 24, 32, stride, handle, &fbId)) return 0;

    // Set destroy callback to auto-cleanup
    gbm_bo_set_user_data(bo, (void *)(uintptr_t)fbId, DestroyFrameBufferCallback);

    TRACELOG(LOG_INFO, "DISPLAY: DRM: Added new buffer object [%u]", (uintptr_t)fbId);

    return fbId;
}

With this change, fbCache, fbCacheCount and MAX_DRM_CACHED_BUFFERS are
no longer used. I left them in place to keep the test small.

Results on the R36S:

• The 4th distinct buffer now receives a framebuffer, and page flips continue
  for the whole session: more than 5700 flips in my test, instead of 2.
• SetTargetFPS(39/60/78) all hold their target. 78 fps matches the 78.2 Hz
  panel and stays there.
• Without a frame limit it also does not freeze: the loop produced about
  600 fps while the display kept updating.
• Visually I could not see tearing or judder, the menus respond, and the
  program exits cleanly.

Two notes: I tested this against the raylib 6.0 sources, but both functions
are byte-identical on master (d46a593). And this is still one device and one
driver, so it only shows the approach works on the Mali blob, not that it is
safe everywhere. I am glad to run any other test that would help.

[rob-bits]

Ohh, and I forgot. Enabling the SUPPORT_CUSTOM_FRAME_CONTROL will remove the timing of the swap screen buffer. The application should maintain proper timing. In the main loop I used the following logic:

    static struct timespec gNext_frame_time = clock_gettime(CLOCK_MONOTONIC, &gNext_frame_time);
    struct timespec currTime;
    static long frame_ns = 16666667; // 16.666667 ms for 60Hz
    //static long frame_ns = 8333333; // 8.333333 ms for 120Hz
    clock_gettime(CLOCK_MONOTONIC, &currTime);
    
    long current_sec = currTime.tv_sec;
    long target_sec = gNext_frame_time.tv_sec;
    long current_ns = currTime.tv_sec * 1000000000L + currTime.tv_nsec;
    long target_ns = gNext_frame_time.tv_sec * 1000000000L + gNext_frame_time.tv_nsec;

    if (current_ns >= target_ns) {
        // Calculate how many frames we missed
        long missed_frames = (current_ns - target_ns) / frame_ns;
        // Set next frame time (skip missed frames)
        target_ns += (missed_frames + 1) * frame_ns;
        gNext_frame_time.tv_sec = target_ns / 1000000000L;
        gNext_frame_time.tv_nsec = target_ns % 1000000000L;

    //... updating textures
        BeginDrawing();
    //drawing textures
        EndDrawing();

        SwapScreenBuffer();                  // Copy back buffer to front buffer (screen)
        PollInputEvents();      // Poll user events once in every second
    }

And my application was multi-threaded. One high priority, real-time thread, a specific core were assigned to the Raylib related code. The time for updating and drawing of the textures also critical. If you miss your target frametime then the buffer caching will not help.

[jtakakura]

Thank you for the explanation. Your timing loop makes sense to me, and I understand that with SUPPORT_CUSTOM_FRAME_CONTROL the application must keep the frame timing itself.

I would like to explain why I believe frame timing is not the cause of the freeze on this device:

• The freeze also happens with the default configuration, where EndDrawing()
  performs the normal SetTargetFPS(60) timing. At that moment the scene is
  almost empty (a title screen), and producing an empty frame takes about 1 ms
  on this device, so the 16.6 ms deadline is never missed. It still freezes
  within the first frames: only two page flips ever happen.
• The 4th distinct buffer appears at the very start of the session in every
  configuration I tried, fast or slow. It does not appear after a slow frame.
  On this driver it is simply what gbm_surface_lock_front_buffer() returns.
• With the user data patch from my previous comment I tested both extremes:
  SetTargetFPS(78), which holds the deadline on the 78.2 Hz panel, and no
  frame limit at all. Neither one freezes, and I did not use any thread
  priority or core pinning.

So on the Mali driver the deciding factor seems to be that the surface rotates
four buffers while fbCache can hold three.

May I ask one thing about your setup? On the Raspberry Pi, does gbm_surface_lock_front_buffer() ever return a 4th distinct buffer pointer?
If mesa rotates only three, the array never overflows there, and that would explain why our results look so different.

[raysan5]

@jtakakura @rob-bits thanks for the quick review of this issue.

Please note that SUPPORT_CUSTOM_FRAME_CONTROL is disabled by default and shouldn't be a requirement for SUPPORT_DRM_CACHE to work properly.

[rob-bits]

In my application log, I see:

INFO: DRM: added new bo 671
INFO: DRM: added new bo 670
INFO: DRM: added new bo 673

So I got only three buffers. Btw, I am still on Raylib 5.6 version. I implemented the cache buffering like a year ago. Still daily using this part, so did not worked on anymore. I have to look into the code to get the idea. The data patch might be okay, I can test this soon or later. :)

Btw, the errCnt were used for debuging:

errCnt[0] -> pendingFlip error, it misses a frame
errCnt[1] -> could not lock the front buffer, if there the CPU uses the RAM with high bandwidth, the GPU might lag to get the data
errCnt[2] -> frame buffer related, failed to get the buffer, it is similar to previous issue
errCnt[3] -> display is busy, might still drawing the screen from previous FB
errCnt[4] -> page flip failed for any reason
loopCnt -> with 60fps, it should be printed steady 600

I found some old log from 2026.01.27, I see the following values:

INFO: DRM err counters: 239, 0, 0, 0, 0, 600
INFO: DRM err counters: 250, 0, 0, 0, 0, 601
INFO: DRM err counters: 242, 0, 0, 0, 0, 600
INFO: DRM err counters: 260, 0, 0, 0, 0, 600
INFO: DRM err counters: 257, 0, 0, 0, 0, 601
INFO: DRM err counters: 243, 0, 0, 0, 0, 600

errCnt 1-4 were always 0. The errCnt 0 could be not zero, if the timing is proper or if the frame time is missed. For loopCnt I saw 600-601. Regardsless of these measure, my application was responsive, and steady.

And indeed the SUPPORT_CUSTOM_FRAME_CONTROL is a dev feature, might have bugs and it is not generalized much. It was fine tuned for pi zero 2w. So maybe your data patch might be better. But I have to read after and check how this is happening on a pi, or if it needs any generalization.

[jtakakura]

Thank you for checking your log, and for explaining the counters. That answers my question clearly: your Pi rotates three buffers, this device rotates four.

Using your counter names, my case looks like this:

• errCnt[2] (failed to get the framebuffer) increases on almost every frame,
  from the 4th buffer onwards. In your log it is always 0.
• errCnt[0] (pending flip) stays low here, so frames are not being missed.
• errCnt[3] and errCnt[4] never happen. The display is not busy and no
  page flip is rejected. Only two flips are ever issued.

So the same counter that is always 0 on the Pi is the one that never stops
increasing here. I think this is only because MAX_DRM_CACHED_BUFFERS is 3
and this driver returns a 4th buffer. On mesa the array is never full, so the
same code path is never reached.

Please take your time with the review. There is no hurry from my side: I can
use the default DRM path in the meantime, and the patch works for me locally.
If it helps, I am happy to test any version you prefer on this hardware,
including one that keeps the array and only grows or evicts it, if you would rather not change the structure.