Rpi4: hwdec=v4l2m2m is slower than default hwdec=v4l2m2m

Important Information

-mpv version: latest GIT code
-Debian 11 aarch64
-Source of the mpv binary: built from latest GIT source
-Window Manager and version: latest stable labwc Wayland compositor (0.6.3)
-GPU driver and version: mesa-23.1.0 v3d (Raspberry Pi 4)
-ffmpeg version: https://github.com/jc-kynesim/rpi-ffmpeg/tree/dev/6.0/rpi_import_1
(Also branch release/5.1/main has been tried with the same results).

Reproduction steps

-Run mpv on Wayland with:

hwdec=v4l2m2m
video-sync=display-resample

That causes constant frame drops.

-Run mpv on Wayland using:

hwdec=v4l2m2m-copy
video-sync=display-resample

That causes smooth playback.

Expected behavior

hwdec=v4l2m2m and hwdec=v4l2m2m-copy should both work well, no frame drops for 1080p content, or at least hwdec=v4l2m2m should be faster than hwdec=v4l2m2m-copy.

Actual behavior

hwdec=v4l2m2m causes frame drops and hwdec=v4l2m2m-copy does not.
That makes not much sense, because CPU usage is actually LOWER with hwdec=v4l2m2m.

Log file

This is the log with

hwdec=v4l2m2m
video-sync=display-resample

(There are dropped frames even if CPU usage is lower)
log1.txt

This is the log with

hwdec=v4l2m2m-copy
video-sync=display-resample

(No dropped frames even if CPU usage is higher).
log2.txt

Start by reproducing the two Wayland configurations on the Raspberry Pi 4 and compare log1.txt with log2.txt, focusing on frame drops and timing. The issue is done when hwdec=v4l2m2m provides smooth 1080p playback like hwdec=v4l2m2m-copy, or the performance difference is explained and documented.