video_encode:21:RIL: no space in FIFO for headers in vcdbg while using v4l2 h264 encode with Repeat Sequence Headers

Start with static int job_ready(void *priv) in bcm2835-v4l2-codec.c and compare the good/bad dmesg and vcdbg logs linked in the report. Reproduce the issue using the ffmpeg-4.4 v4l2_m2m_enc.c change that enables REPEAT_SEQ_HEADER, then trace buffer and header handling. Done means repeated sequence headers no longer produce FIFO errors or leave all output buffers queued.

Is this the right place for my bug report?
I believe it is associated with the v4l encode device.

Describe the bug
video_encode:21:RIL: no space in FIFO for headers is printed in sudo vcdbg log msg with vcgencmd set_logging level=0x400100C0 while encoding with REPEAT_SEQ_HEADER set to 1.

The error message does not appear initially, but consistently after ~20000 frames encoded. The timing of when it appears doesn't appear to be too spread out
The error message also does not appear if REPEAT_SEQ_HEADER=0

While using ffmpeg in code, frame write and packet read also became inconsistent after a while, which I traced back to all(16) output (data going to driver) buffers being in driver after a certain period which coincided with the error messages being produced. There were few outstanding capture (from driver) buffers, i.e the problem was not lack of capture buffers in driver.

Note: I am using ffmpeg only to generate and encode the video, I do not believe it is a problem with ffmpeg.

To reproduce
get ffmpeg-4.4 release
modify v4l2_m2m_enc.c to include and set REPEAT_SEQ_HEADER to 1. as seen in this patch which was found: here
build ffmpeg (any compile flags should be good I have attached the used ffmpeg -version)

run vcgencmd set_logging level=0x400100C0
run ffmpeg -re -loglevel info -f lavfi -i color=size=1408x1086:rate=30 -codec h264_v4l2m2m -pix_fmt yuv420p -b:v 8M -f null -
run sudo vcdbg log msg and eventually the errors start appearing

ffmpeg_version.txt

Expected behaviour
No error messages and no backlog of output(to driver) buffers

Actual behaviour
Error messages and all output buffers stored in driver

System
https://pastebin.com/37j7k6E2

I have removed dmesg and vcdbg from raspinfo as i will attach the relevant parts below. Let me know if you need this too (it is the same I believe)

Logs
I have created logs of dmesg and vcdbg log msg they are named good/bad for when REPEAT_SEQ_HEADER is enabled/disabled respectively.
For dmesg echo Y > /sys/module/v4l2_mem2mem/parameters/debug was set.
For 'bad' vcdbg i have created a sample of the logs at ~5000 frame intervals

dmesgbad.txt
dmesggood.txt
vcdbgloggood.txt
vcdbglogbad5000.txt
vcdbglogbad10000.txt
vcdbglogbad15000.txt
vcdbglogbad20000.txt
vcdbglogbad25000.txt

Additional context
I believe it is to do with job_ready only allowing a job to proceed if both src and dest buffers are available, in specific in
static int job_ready(void *priv) in bcm2835-v4l2-codec.c

When testing it in code I had a breakpoint set for if all output buffers fill. When this breakpoint was hit video_encode:21:RIL: no space in FIFO for headers was printed continuously while the program was paused with no intermediate messages(vcgencmd set_logging level=0x400100**4**0) I have logs of that too if needed.

The weird size and static colour are to replicate the images I was using when I noticed it. The width was also chosen to be 64 byte aligned. see https://github.com/raspberrypi/linux/pull/4419 which I believe affects bcm2835-v4l2-codec too (seperate issue)