[5.4.59-v8+] bcm2835-v4l2: Failed to enable encode tunnel - error -2

Start with the bcm2835_v4l2 driver and the vb2_start_streaming path shown in the trace, then reproduce H.264 streaming with v4l2rtspserver on the reported kernel. Done means the encode tunnel starts without the reported error or videobuf2 warning under the supplied setup.

Hi,

When running v4l2rtspserver on the latest kernel I'm getting the following error in dmesg:

[ 8792.227039] bcm2835-v4l2: Failed to enable encode tunnel - error -2
[ 8792.227138] ------------[ cut here ]------------
[ 8792.227185] WARNING: CPU: 0 PID: 2925 at drivers/media/common/videobuf2/videobuf2-core.c:1487 vb2_start_streaming+0xdc/0x158 [videobuf2_common]
[ 8792.227211] Modules linked in: ipv6 nf_defrag_ipv6 bcm2835_v4l2(C) bcm2835_mmal_vchiq(C) vc_sm_cma(C) videobuf2_vmalloc videobuf2_memops videobuf2_v4l2 videobuf2_common videodev mc
[ 8792.227275] CPU: 0 PID: 2925 Comm: v4l2rtspserver Tainted: G        WC        5.4.59-v8+ #1338
[ 8792.227297] Hardware name: Raspberry Pi 4 Model B Rev 1.1 (DT)
[ 8792.227316] pstate: 60000005 (nZCv daif -PAN -UAO)
[ 8792.227344] pc : vb2_start_streaming+0xdc/0x158 [videobuf2_common]
[ 8792.227371] lr : vb2_start_streaming+0x6c/0x158 [videobuf2_common]
[ 8792.227388] sp : ffffffc015e73a50
[ 8792.227401] x29: ffffffc015e73a50 x28: 0000000000000001
[ 8792.227422] x27: ffffff80dc623100 x26: ffffffc008a9b238
[ 8792.227440] x25: ffffff80dc623100 x24: ffffff80dc5ffb00
[ 8792.227459] x23: ffffff80f5a33590 x22: 00000000ffffffff
[ 8792.227478] x21: ffffff80f5a33840 x20: ffffff80f5a33a18
[ 8792.227496] x19: ffffff80f5a33838 x18: ffffffc010e58888
[ 8792.227515] x17: 0000000000000000 x16: 0000000000000000
[ 8792.227533] x15: ffffffc095e73737 x14: 0720072007200720
[ 8792.227551] x13: 0720072007200720 x12: 072007200732072d
[ 8792.227569] x11: 07200772076f0772 x10: 077207650720072d
[ 8792.227587] x9 : 0720076c0765076e x8 : 00000000000001fd
[ 8792.227606] x7 : 0000000000000000 x6 : ffffffc010e59000
[ 8792.227624] x5 : ffffffc010e59058 x4 : 0000000000000001
[ 8792.227642] x3 : ffffffc01014adfc x2 : 53d839196b567600
[ 8792.227660] x1 : 0000000000000000 x0 : 000000000000000a
[ 8792.227678] Call trace:
[ 8792.227702]  vb2_start_streaming+0xdc/0x158 [videobuf2_common]
[ 8792.227728]  vb2_core_streamon+0x178/0x1a0 [videobuf2_common]
[ 8792.227757]  vb2_streamon+0x54/0x78 [videobuf2_v4l2]
[ 8792.227780]  vb2_ioctl_streamon+0x58/0x68 [videobuf2_v4l2]
[ 8792.227836]  v4l_streamon+0x3c/0x50 [videodev]
[ 8792.227886]  __video_do_ioctl+0x234/0x3f0 [videodev]
[ 8792.227936]  video_usercopy+0x274/0x6b0 [videodev]
[ 8792.227985]  video_ioctl2+0x3c/0x78 [videodev]
[ 8792.228034]  v4l2_ioctl+0x64/0x90 [videodev]
[ 8792.228062]  do_vfs_ioctl+0xc8/0xa08
[ 8792.228081]  ksys_ioctl+0x50/0x80
[ 8792.228100]  __arm64_sys_ioctl+0x28/0xb8
[ 8792.228121]  el0_svc_common.constprop.1+0x7c/0x1b0
[ 8792.228139]  el0_svc_handler+0x34/0xa0
[ 8792.228156]  el0_svc+0x8/0x204
[ 8792.228171] ---[ end trace 1404d3fc5552aabc ]---

Kernel:

$ uname -a
Linux localhost 5.4.59-v8+ #1338 SMP PREEMPT Fri Aug 28 12:02:32 BST 2020 aarch64 Linux

Loaded modules:

$  lsmod
Module                  Size  Used by    Tainted: G
ipv6                  491520 18 [permanent]
nf_defrag_ipv6         20480  1 ipv6
bcm2835_v4l2           45056  0
bcm2835_mmal_vchiq     32768  1 bcm2835_v4l2
vc_sm_cma              40960  1 bcm2835_mmal_vchiq
videobuf2_vmalloc      20480  1 bcm2835_v4l2
videobuf2_memops       16384  1 videobuf2_vmalloc
videobuf2_v4l2         28672  1 bcm2835_v4l2
videobuf2_common       57344  2 bcm2835_v4l2,videobuf2_v4l2
videodev              270336  3 bcm2835_v4l2,videobuf2_v4l2,videobuf2_common
mc                     53248  3 videobuf2_v4l2,videobuf2_common,videodev

V4L2:

$ v4l2-ctl --all
Driver Info:
	Driver name      : bm2835 mmal
	Card type        : mmal service 16.1
	Bus info         : platform:bcm2835-v4l2
	Driver version   : 5.4.59
	Capabilities     : 0x85200005
		Video Capture
		Video Overlay
		Read/Write
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x05200005
		Video Capture
		Video Overlay
		Read/Write
		Streaming
		Extended Pix Format
Priority: 2
Video input : 0 (Camera 0: ok)
Format Video Capture:
	Width/Height      : 1920/1080
	Pixel Format      : 'H264' (H.264)
	Field             : None
	Bytes per Line    : 0
	Size Image        : 2088960
	Colorspace        : SMPTE 170M
	Transfer Function : Default (maps to Rec. 709)
	YCbCr/HSV Encoding: Default (maps to ITU-R 601)
	Quantization      : Default (maps to Full Range)
	Flags             :
Format Video Overlay:
	Left/Top    : 150/50
	Width/Height: 1024/768
	Field       : None
	Chroma Key  : 0x00000000
	Global Alpha: 0xff
	Clip Count  : 0
	Clip Bitmap : No
Framebuffer Format:
	Capability    : Extern Overlay
			Global Alpha
	Flags         : Overlay Matches Capture/Output Size
	Width         : 1920
	Height        : 1088
	Pixel Format  : 'YU12'
Streaming Parameters Video Capture:
	Capabilities     : timeperframe
	Frames per second: 25.000 (25/1)
	Read buffers     : 1

User Controls

                     brightness 0x00980900 (int)    : min=0 max=100 step=1 default=50 value=50 flags=slider
                       contrast 0x00980901 (int)    : min=-100 max=100 step=1 default=0 value=0 flags=slider
                     saturation 0x00980902 (int)    : min=-100 max=100 step=1 default=0 value=0 flags=slider
                    red_balance 0x0098090e (int)    : min=1 max=7999 step=1 default=1000 value=1000 flags=slider
                   blue_balance 0x0098090f (int)    : min=1 max=7999 step=1 default=1000 value=1000 flags=slider
                horizontal_flip 0x00980914 (bool)   : default=0 value=0
                  vertical_flip 0x00980915 (bool)   : default=0 value=0
           power_line_frequency 0x00980918 (menu)   : min=0 max=3 default=1 value=1
				0: Disabled
				1: 50 Hz
				2: 60 Hz
				3: Auto
                      sharpness 0x0098091b (int)    : min=-100 max=100 step=1 default=0 value=0 flags=slider
                  color_effects 0x0098091f (menu)   : min=0 max=15 default=0 value=0
				0: None
				1: Black & White
				2: Sepia
				3: Negative
				4: Emboss
				5: Sketch
				6: Sky Blue
				7: Grass Green
				8: Skin Whiten
				9: Vivid
				10: Aqua
				11: Art Freeze
				12: Silhouette
				13: Solarization
				14: Antique
				15: Set Cb/Cr
                         rotate 0x00980922 (int)    : min=0 max=360 step=90 default=0 value=0 flags=modify-layout
             color_effects_cbcr 0x0098092a (int)    : min=0 max=65535 step=1 default=32896 value=32896

Codec Controls

             video_bitrate_mode 0x009909ce (menu)   : min=0 max=1 default=0 value=0 flags=update
				0: Variable Bitrate
				1: Constant Bitrate
                  video_bitrate 0x009909cf (int)    : min=25000 max=25000000 step=25000 default=10000000 value=10000000
         repeat_sequence_header 0x009909e2 (bool)   : default=0 value=0
            h264_i_frame_period 0x00990a66 (int)    : min=0 max=2147483647 step=1 default=60 value=60
                     h264_level 0x00990a67 (menu)   : min=0 max=11 default=11 value=11
				0: 1
				1: 1b
				2: 1.1
				3: 1.2
				4: 1.3
				5: 2
				6: 2.1
				7: 2.2
				8: 3
				9: 3.1
				10: 3.2
				11: 4
                   h264_profile 0x00990a6b (menu)   : min=0 max=4 default=4 value=4
				0: Baseline
				1: Constrained Baseline
				2: Main
				4: High

Camera Controls

                  auto_exposure 0x009a0901 (menu)   : min=0 max=3 default=0 value=0
				0: Auto Mode
				1: Manual Mode
         exposure_time_absolute 0x009a0902 (int)    : min=1 max=10000 step=1 default=1000 value=1000
     exposure_dynamic_framerate 0x009a0903 (bool)   : default=0 value=0
             auto_exposure_bias 0x009a0913 (intmenu): min=0 max=24 default=12 value=12
				0: -4000 (0xfffffffffffff060)
				1: -3667 (0xfffffffffffff1ad)
				2: -3333 (0xfffffffffffff2fb)
				3: -3000 (0xfffffffffffff448)
				4: -2667 (0xfffffffffffff595)
				5: -2333 (0xfffffffffffff6e3)
				6: -2000 (0xfffffffffffff830)
				7: -1667 (0xfffffffffffff97d)
				8: -1333 (0xfffffffffffffacb)
				9: -1000 (0xfffffffffffffc18)
				10: -667 (0xfffffffffffffd65)
				11: -333 (0xfffffffffffffeb3)
				12: 0 (0x0)
				13: 333 (0x14d)
				14: 667 (0x29b)
				15: 1000 (0x3e8)
				16: 1333 (0x535)
				17: 1667 (0x683)
				18: 2000 (0x7d0)
				19: 2333 (0x91d)
				20: 2667 (0xa6b)
				21: 3000 (0xbb8)
				22: 3333 (0xd05)
				23: 3667 (0xe53)
				24: 4000 (0xfa0)
      white_balance_auto_preset 0x009a0914 (menu)   : min=0 max=10 default=1 value=1
				0: Manual
				1: Auto
				2: Incandescent
				3: Fluorescent
				4: Fluorescent H
				5: Horizon
				6: Daylight
				7: Flash
				8: Cloudy
				9: Shade
				10: Greyworld
            image_stabilization 0x009a0916 (bool)   : default=0 value=0
                iso_sensitivity 0x009a0917 (intmenu): min=0 max=4 default=0 value=0
				0: 0 (0x0)
				1: 100000 (0x186a0)
				2: 200000 (0x30d40)
				3: 400000 (0x61a80)
				4: 800000 (0xc3500)
           iso_sensitivity_auto 0x009a0918 (menu)   : min=0 max=1 default=1 value=1
				0: Manual
				1: Auto
         exposure_metering_mode 0x009a0919 (menu)   : min=0 max=2 default=0 value=0
				0: Average
				1: Center Weighted
				2: Spot
                     scene_mode 0x009a091a (menu)   : min=0 max=13 default=0 value=0
				0: None
				8: Night
				11: Sports

JPEG Compression Controls

            compression_quality 0x009d0903 (int)    : min=1 max=100 step=1 default=30 value=30