VA_DRM_GetNumCandidates changed behaviour, therefore aborts X server in VM with QXL driver.
- Dominant language
- C
- Stars
- 784
- Forks
- 314
- PR merge metrics
- No merged PRs in 30d
Description
In current Debian Testing running inside a qemu VM with the QML driver the X server can be crashed by just running `gst-play-1.0 no-such-file.avi`.
As far as I see `VA_DRM_GetNumCandidates` does compare the driver name "qxl" against a known list of drivers `g_driver_name_map`.
In my debugging this list contains "qxl" neither in libva version 2.17.0-1 (Debian Bookworm/testing)
nor in 2.10.0-1 (Debian Bullseye/stable).
In version 2.10.0 `VA_DRM_GetNumCandidates` returns VA_STATUS_ERROR_UNKNOWN (because count==0).
But in version 2.17.0 it returns unconditionally VA_STATUS_SUCCESS.
It look like this got introduced with commit 317c0fbea5.
Therefore `va_DisplayContextGetNumCandidates` does not return early but does enter `va_drm_authenticate`/`VA_DRI2Authenticate` which finally causes the X server to hit an assertion and ends therefore.
Because I guess the X server should be resistant even for such queries
I opened https://gitlab.freedesktop.org/xorg/xserver/-/issues/1534.
But libva probably want to either revert back to original behaviour of `VA_DRM_GetNumCandidates`,
or wants to change users of it to check for the returned num_candidates > 0 or something like this?
https://github.com/intel/libva/blob/0fa448dd00c509f08cf53b9a95a55922ff09d5cf/va/drm/va_drm.c#L54-L56
```
(gdb) bt
#0 VA_DRI2Authenticate (dpy=dpy@entry=0x555555625aa0, window=1338, magic=magic@entry=1) at ./va/x11/va_dri2.c:206
#1 0x00007ffff70b0722 in va_drm_authenticate_x11 (fd=fd@entry=6, magic=magic@entry=1) at ./va/drm/va_drm_auth_x11.c:140
#2 0x00007ffff70b041f in va_drm_authenticate (fd=6, magic=1) at ./va/drm/va_drm_auth.c:37
#3 0x00007ffff70b02df in va_DisplayContextGetNumCandidates (pDisplayContext=, num_candidates=) at ./va/drm/va_drm.c:73
#4 0x00007ffff725cedb in vaInitialize () from /lib/x86_64-linux-gnu/libva.so.2
#5 0x00007ffff72fa878 in vaapi_initialize (dpy=dpy@entry=0x555555624e10) at ../gst-libs/gst/vaapi/gstvaapiutils.c:113
#6 0x00007ffff7322894 in supports_vaapi (fd=6) at ../gst-libs/gst/vaapi/gstvaapidisplay_drm.c:77
#7 get_default_device_path (display=0x5555556221d0) at ../gst-libs/gst/vaapi/gstvaapidisplay_drm.c:140
#8 set_device_path (device_path=, display=0x5555556221d0) at ../gst-libs/gst/vaapi/gstvaapidisplay_drm.c:181
#9 gst_vaapi_display_drm_open_display (display=0x5555556221d0, name=) at ../gst-libs/gst/vaapi/gstvaapidisplay_drm.c:247
#10 0x00007ffff72ef544 in gst_vaapi_display_create (data=0x0, init_type=GST_VAAPI_DISPLAY_INIT_FROM_DISPLAY_NAME, display=0x5555556221d0) at ../gst-libs/gst/vaapi/gstvaapidisplay.c:965
#11 gst_vaapi_display_config (display=0x5555556221d0, init_type=init_type@entry=GST_VAAPI_DISPLAY_INIT_FROM_DISPLAY_NAME, init_value=init_value@entry=0x0) at ../gst-libs/gst/vaapi/gstvaapidisplay.c:1272
#12 0x00007ffff7322b40 in gst_vaapi_display_drm_new (device_path=0x0) at ../gst-libs/gst/vaapi/gstvaapidisplay_drm.c:367
#13 0x00007ffff72b96d0 in gst_vaapi_create_test_display () at ../gst/vaapi/gstvaapipluginutil.c:929
#14 0x00007ffff72b0899 in plugin_init (plugin=0x5555555ea030) at ../gst/vaapi/gstvaapi.c:191
#15 0x00007ffff7f041a2 in gst_plugin_register_func (plugin=plugin@entry=0x5555555ea030, desc=desc@entry=0x7ffff736f680 , user_data=user_data@entry=0x0) at ../gst/gstplugin.c:532
#16 0x00007ffff7f062ef in _priv_gst_plugin_load_file_for_registry (filename=filename@entry=0x555555562bcc "/lib/x86_64-linux-gnu/gstreamer-1.0/libgstvaapi.so", registry=0x55555556d960, registry@entry=0x0, error=error@entry=0x0) at ../gst/gstplugin.c:971
#17 0x00007ffff7f06baa in gst_plugin_load_file (filename=filename@entry=0x555555562bcc "/lib/x86_64-linux-gnu/gstreamer-1.0/libgstvaapi.so", error=error@entry=0x0) at ../gst/gstplugin.c:689
#18 0x00007ffff7f093f2 in do_plugin_load (tag=12, filename=0x555555562bcc "/lib/x86_64-linux-gnu/gstreamer-1.0/libgstvaapi.so", l=0x55555556cc30) at ../gst/gstpluginloader.c:848
#19 handle_rx_packet (payload_len=, payload=0x555555562bcc "/lib/x86_64-linux-gnu/gstreamer-1.0/libgstvaapi.so", tag=12, pack_type=, l=0x55555556cc30) at ../gst/gstpluginloader.c:956
#20 read_one (l=0x55555556cc30) at ../gst/gstpluginloader.c:1132
#21 exchange_packets (l=l@entry=0x55555556cc30) at ../gst/gstpluginloader.c:1160
#22 0x00007ffff7f0a708 in _gst_plugin_loader_client_run () at ../gst/gstpluginloader.c:700
#23 0x0000555555555165 in main (argc=, argv=0x7fffffffe3c8) at ../libs/gst/helpers/gst-plugin-scanner.c:67
```
Contributor guide
Assessment
This issue has not been assessed yet.