Pi 5 HEVC: rpi-hevc-dec returns EINVAL on VIDIOC_S_EXT_CTRLS, and v4l2slh265dec then segfaults, when two consumers share the decoder's zero-copy frames

Start by tracing the VIDIOC_S_EXT_CTRLS path in rpi-hevc-dec alongside gst_v4l2_decoder_set_controls and gst_v4l2_codec_h265_dec_submit_bitstream. Use the two-consumer GTK/GLES reproduction and the supplied GST_DEBUG output, then inspect gst_v4l2_codec_h265_dec_output_picture. Done means the refused control operation no longer leads to a process crash on the shared zero-copy path.

Describe the bug

On a Pi 5, hardware HEVC decoding through v4l2slh265dec takes the whole process down with SIGSEGV one to three seconds after a second consumer starts taking frames from the same decoder. With a single consumer the identical pipeline decodes for hours (>350k frames in one run).

Two things happen, in this order:

1. rpi-hevc-dec refuses exactly one VIDIOC_S_EXT_CTRLS with EINVAL, out of a completely regular decode loop. It happens immediately after a capture buffer is handed back by the downstream thread and re-armed by the decode thread ~260 µs later — a sequence that only occurs once a second, slower consumer is attached.
2. v4l2slh265dec posts its "Driver did not accept the bitstream parameters" error and then continues into gst_v4l2_codec_h265_dec_output_picture instead of stopping — and dies there. So a refused ioctl ends the application rather than the stream.

I am reporting this here rather than with GStreamer because both halves ship in your image: the driver is yours, and the plugin is your build (1.26.2-3+rpt3+deb13u2). Please forward the plugin half upstream if you think it belongs there.

Steps to reproduce the behaviour

1. Decode a 1280x720 HEVC stream (coded 1280x736 with a conformance window) at 60 fps with v4l2slh265dec and import the frames into GL:
   appsrc → h265parse → decodebin3 → glupload → glcolorconvert → tee
2. Branch 1: queue → glvideoflip → gtkglsink — this runs indefinitely.
3. Branch 2, opened at runtime: queue → glvideoflip → gldownload → videoconvert → gtksink.
4. Crash within 1–3 seconds, every time.

Not reproducible with gst-launch + fakesink: the decoder then falls back to copies (GstVideoMeta support required, copying frames), so the zero-copy DMABuf path this happens on is never taken. A faithful reproduction needs a real GTK/GLES consumer — ours is an open-source application (GPL-3.0) and I can point at the source or build a stripped-down test case if that helps.

Device(s)

Raspberry Pi 5

System

Image:    Raspberry Pi reference 2026-06-18 (pi-gen ca8aeed0, stage4), Debian 13 trixie, 64-bit
Kernel:   6.18.48-v8-16k+
Firmware: f5e89631 (2026/08/12)
Driver:   rpi-hevc-dec 6.18.48 (/dev/video19)
GStreamer 1.26.2 · libgstreamer-plugins-bad1.0-0 1.26.2-3+rpt3+deb13u2
Mesa      26.2.1-2~bpo13+0~rpt1

Logs

GST_DEBUG='*v4l2*:6,GST_BUFFER_POOL:5,videodecoder:4' — 464 regular allocate/output/release cycles before this, no pool starvation, no renegotiation:

allocator_release:<v4l2codecallocator1> Placing back buffer 2 into pool   (downstream thread)
allocator_prepare_buffer:<v4l2codecallocator1> Allocated buffer 2         (decode thread, +260 µs)
ERROR gstv4l2decoder.c:1051:gst_v4l2_decoder_set_controls: VIDIOC_S_EXT_CTRLS failed: Invalid argument
WARN  gstv4l2codech265dec.c:1465:gst_v4l2_codec_h265_dec_submit_bitstream: error: Driver did not accept the bitstream parameters.
DEBUG gstv4l2codech265dec.c:1276:gst_v4l2_codec_h265_dec_output_picture: Output picture 465
<SIGSEGV>

Backtrace (release build, plugin unsymbolised):

Thread "multiqueue0:src" received signal SIGSEGV
#0,#1  libgstv4l2codecs.so
#2-#4  libgstcodecs-1.0.so.0
#5-#7  libgstvideo-1.0.so.0
#8,#9  libgstreamer-1.0.so.0
#10    gst_pad_push
#11    libgstcoreelements.so (multiqueue)

Additional context

CMA was ruled out — sampled at 2 Hz through a reproduction, CmaFree never moved from 54 MB of 64 MB.

The same application code on x86 with VA-API (vah265dec) never shows this, which is why it took a Pi to find it.

Our workaround is to stop letting two consumers share one decoder on this hardware: every surface gets its own pipeline, i.e. a second hardware decode. That is stable.