Low bitrate allocation for first ~10 frames with FFmpeg h264_v4l2m2m encoder

Start by reproducing the FFmpeg h264_v4l2m2m command against /dev/shm/camera.yuv and comparing its packet sizes with the GStreamer and libx264 results. Use the supplied Compute Module 4, kernel details, and system-information link to investigate whether the first-frame allocation behavior is in the kernel encoder path; done means identifying a reproducible cause and a verified correction or clear upstream report.

This very well could be more of an FFmpeg issue, but since I can't replicate it with FFmpeg's libx264 or Gstreamer's v4l2h264enc, I figured it would be worth posting for investigation. And to be fair, most people probably wouldn't care about those first ~10 frames (ie. of a movie), but our use-case requires even quality across all frames since our clients may pause on each frame (limited to about 300 frames, and played in a loop).

After capturing a YUV video stream from the Pi HQ camera via: libcamera-vid --width 4096 --height 3040 -t 24000 --gain 4 --awbgains 1,1 --shutter 100000 --denoise cdn_fast --codec yuv420 -o /dev/shm/camera.yuv, I then encoded the YUV stream three ways:

FFmpeg h264_v4l2m2m via ffmpeg -y -f rawvideo -pix_fmt yuv420p -s:v 4096x3040 -r 1 -i /dev/shm/camera.yuv -vf crop=1024:1520:0:0 -c:v h264_v4l2m2m -b:v 25M ~/h264_v4l2m2m.mp4 results in the following frame sizes (starting with the first frame):

pkt_size=6135
pkt_size=1039
pkt_size=805
pkt_size=1936
pkt_size=4071
pkt_size=6785
pkt_size=15970
pkt_size=32366
pkt_size=54912
pkt_size=91147
pkt_size=108113
pkt_size=107580
pkt_size=107819
pkt_size=107843
pkt_size=108054
pkt_size=108166
pkt_size=107747
pkt_size=108400
pkt_size=108100
pkt_size=107681
pkt_size=107593

Gstreamer v4l2h264enc via gst-launch-1.0 filesrc location=/dev/shm/camera.yuv ! rawvideoparse use-sink-caps=false width=4096 height=3040 format=i420 framerate=1/1 ! videocrop top=0 left=0 right=3072 bottom=1520 ! v4l2convert ! v4l2h264enc extra-controls="controls,video_bitrate_mode=0,video_bitrate=25000000,h264_i_frame_period=1000,repeat_sequence_header=1" ! 'video/x-h264,profile=high,level=(string)4.2' ! h264parse ! mp4mux ! filesink location=~/v4l2h264enc.mp4

pkt_size=516362
pkt_size=113457
pkt_size=111495
pkt_size=111918
pkt_size=111882
pkt_size=111977
pkt_size=112440
pkt_size=112394
pkt_size=112398
pkt_size=112171
pkt_size=112389
pkt_size=111820

FFmpeg libx264 via ffmpeg -y -f rawvideo -pix_fmt yuv420p -s:v 4096x3040 -r 1 -i /dev/shm/camera.yuv -vf crop=1024:1520:0:0 -c:v libx264 -b:v 25M ~/libx264.mp4

pkt_size=1027409
pkt_size=1025456
pkt_size=1024378
pkt_size=1025165
pkt_size=1024789
pkt_size=1025044
pkt_size=1024610
pkt_size=1025006
pkt_size=1025209
pkt_size=1025857

I am testing on a Compute Module 4 + Official IO Board + Bullseye Lite 64bit + 5.10.63-v8+ Kernel. Full system info here - https://pastebin.com/654vBepL

Let me know if you need any further info or help replicating.