Green artifact / black screen on HEVC video

Important Information

• mpv version = 0.32.0
• Linux Distribution and Version = Debian sid (dist-upgrade on 18/04/2020)
• Kernel version = 5.5.0.1
• Source of the mpv binary = Offical debian repo
• If known which version of mpv introduced the problem = no, but this video worked fine on debian sid / mpv 0.32.0 / nvidia card
• Window Manager and version = xfwm4 4.14.0
• GPU driver and version = AMD 5700 XT Special Edition with driver 20.0.4
• Possible screenshot or video of visual glitches = See samples

Reproduction steps

Launch mpv with default mpv.conf and one of those param on a hevc video :
hwdec=vdpau
hwdec=vaapi

Expected behavior

Able to use hardware decoding

Actual behavior

With hwdec=vaapi i get green artifact on all the image with the first video (but it work with vdpau)
With hwdec=vdpau i get black screen on the second video (but it work with vaapi)

Log file

Log.Video.Green.Artificact.with.VAAPI.txt
Log.Video.Black.Screen.with.VDPAU.txt

Sample files

I can't provide video samples since it's copyrighted material. But here is the screenshot for the Green artifact (the black screen one is just...you know) :

https://imgur.com/a/fCNmRGU

Here is the mediainfo for the green artifact :

General
Format                                   : Matroska
Format version                           : Version 4
File size                                : 8.75 MiB
Duration                                 : 30 s 130 ms
Overall bit rate                         : 2 435 kb/s
Writing application                      : Lavf58.29.100
Writing library                          : Lavf58.29.100
ErrorDetectionType                       : Per level 1

Video
ID                                       : 1
Format                                   : HEVC
Format/Info                              : High Efficiency Video Coding
Format profile                           : Main 10@L3@Main
Codec ID                                 : V_MPEGH/ISO/HEVC
Duration                                 : 30 s 130 ms
Bit rate                                 : 1 499 kb/s
Width                                    : 720 pixels
Height                                   : 480 pixels
Display aspect ratio                     : 4:3
Frame rate mode                          : Variable
Frame rate                               : 1 306.837 FPS
Original frame rate                      : 29.970 (30000/1001) FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 10 bits
Bits/(Pixel*Frame)                       : 0.003
Stream size                              : 235 MiB
Writing library                          : x265 2.9:[Windows][GCC 8.2.0][64 bit] 10bit
Encoding settings                        : cpuid=1111039 / frame-threads=3 / numa-pools=8 / wpp / no-pmode / no-pme / no-psnr / no-ssim / log-level=2 / input-csp=1 / input-res=720x480 / interlace=0 / total-frames=0 / level-idc=0 / high-tier=1 / uhd-bd=0 / ref=3 / no-allow-non-conformance / no-repeat-headers / annexb / no-aud / no-hrd / info / hash=0 / no-temporal-layers / open-gop / min-keyint=30 / keyint=300 / gop-lookahead=0 / bframes=4 / b-adapt=0 / b-pyramid / bframe-bias=0 / rc-lookahead=15 / lookahead-slices=0 / scenecut=40 / radl=0 / no-intra-refresh / ctu=64 / min-cu-size=8 / no-rect / no-amp / max-tu-size=32 / tu-inter-depth=1 / tu-intra-depth=1 / limit-tu=0 / rdoq-level=0 / dynamic-rd=0.00 / no-ssim-rd / signhide / no-tskip / nr-intra=0 / nr-inter=0 / no-constrained-intra / strong-intra-smoothing / max-merge=2 / limit-refs=3 / no-limit-modes / me=1 / subme=2 / merange=57 / temporal-mvp / weightp / no-weightb / no-analyze-src-pics / deblock=0:0 / sao / no-sao-non-deblock / rd=2 / no-early-skip / rskip / fast-intra / no-tskip-fast / no-cu-lossless / no-b-intra / no-splitrd-skip / rdpenalty=0 / psy-rd=2.00 / psy-rdoq=0.00 / no-rd-refine / no-lossless / cbqpoffs=0 / crqpoffs=0 / rc=abr / bitrate=1500 / qcomp=0.60 / qpstep=4 / stats-write=0 / stats-read=2 / cplxblur=20.0 / qblur=0.5 / ipratio=1.40 / pbratio=1.30 / aq-mode=1 / aq-strength=1.00 / cutree / zone-count=0 / no-strict-cbr / qg-size=32 / no-rc-grain / qpmax=69 / qpmin=0 / no-const-vbv / sar=255 / sar-width / : / sar-height=8:9 / overscan=0 / videoformat=5 / range=0 / colorprim=6 / transfer=1 / colormatrix=6 / chromaloc=0 / display-window=0 / max-cll=0,0 / min-luma=0 / max-luma=1023 / log2-max-poc-lsb=8 / vui-timing-info / vui-hrd-info / slices=1 / no-opt-qp-pps / no-opt-ref-list-length-pps / no-multi-pass-opt-rps / scenecut-bias=0.05 / no-opt-cu-delta-qp / no-aq-motion / no-hdr / no-hdr-opt / no-dhdr10-opt / no-idr-recovery-sei / analysis-reuse-level=5 / scale-factor=0 / refine-intra=0 / refine-inter=0 / refine-mv=0 / no-limit-sao / ctu-info=0 / no-lowpass-dct / refine-mv-type=0 / copy-pic=1 / max-ausize-factor=1.0 / no-dynamic-refine / no-single-sei
Default                                  : Yes
Forced                                   : No
Color range                              : Limited
Color primaries                          : BT.601 NTSC
Transfer characteristics                 : BT.709
Matrix coefficients                      : BT.601

And the mediainfo for the black screen one :

General
Format                                   : Matroska
Format version                           : Version 4
File size                                : 6.06 MiB
Duration                                 : 30 s 200 ms
Overall bit rate                         : 1 685 kb/s
Writing application                      : Lavf58.29.100
Writing library                          : Lavf58.29.100
ErrorDetectionType                       : Per level 1

Video
ID                                       : 1
Format                                   : HEVC
Format/Info                              : High Efficiency Video Coding
Format profile                           : Main@L3@Main
Codec ID                                 : V_MPEGH/ISO/HEVC
Duration                                 : 29 s 160 ms
Bit rate                                 : 1 523 kb/s
Width                                    : 684 pixels
Height                                   : 476 pixels
Display aspect ratio                     : 1.277
Frame rate mode                          : Constant
Frame rate                               : 25.000 FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Bits/(Pixel*Frame)                       : 0.187
Stream size                              : 5.29 MiB (87%)
Writing library                          : x265 3.2.1+1-b5c86a64bbbe:[Windows][GCC 9.2.0][64 bit] 8bit+10bit+12bit
Encoding settings                        : cpuid=1111039 / frame-threads=2 / numa-pools=4 / wpp / no-pmode / no-pme / no-psnr / no-ssim / log-level=2 / input-csp=1 / input-res=684x476 / interlace=0 / total-frames=0 / level-idc=0 / high-tier=1 / uhd-bd=0 / ref=4 / no-allow-non-conformance / no-repeat-headers / annexb / no-aud / no-hrd / info / hash=0 / no-temporal-layers / open-gop / min-keyint=25 / keyint=250 / gop-lookahead=0 / bframes=4 / b-adapt=2 / b-pyramid / bframe-bias=0 / rc-lookahead=25 / lookahead-slices=0 / scenecut=40 / radl=0 / no-splice / no-intra-refresh / ctu=64 / min-cu-size=8 / rect / no-amp / max-tu-size=32 / tu-inter-depth=1 / tu-intra-depth=1 / limit-tu=0 / rdoq-level=2 / dynamic-rd=0.00 / no-ssim-rd / signhide / no-tskip / nr-intra=0 / nr-inter=0 / no-constrained-intra / strong-intra-smoothing / max-merge=3 / limit-refs=3 / limit-modes / me=3 / subme=3 / merange=57 / temporal-mvp / no-hme / weightp / no-weightb / no-analyze-src-pics / deblock=0:0 / sao / no-sao-non-deblock / rd=4 / selective-sao=4 / no-early-skip / rskip / no-fast-intra / no-tskip-fast / no-cu-lossless / no-b-intra / no-splitrd-skip / rdpenalty=0 / psy-rd=2.00 / psy-rdoq=1.00 / no-rd-refine / no-lossless / cbqpoffs=0 / crqpoffs=0 / rc=crf / crf=18.0 / qcomp=0.60 / qpstep=4 / stats-write=0 / stats-read=0 / ipratio=1.40 / pbratio=1.30 / aq-mode=2 / aq-strength=1.00 / cutree / zone-count=0 / no-strict-cbr / qg-size=32 / no-rc-grain / qpmax=69 / qpmin=0 / no-const-vbv / sar=255 / sar-width / : / sar-height=8:9 / overscan=0 / videoformat=5 / range=0 / colorprim=5 / transfer=1 / colormatrix=6 / chromaloc=0 / display-window=0 / cll=0,0 / min-luma=0 / max-luma=255 / log2-max-poc-lsb=8 / vui-timing-info / vui-hrd-info / slices=1 / no-opt-qp-pps / no-opt-ref-list-length-pps / no-multi-pass-opt-rps / scenecut-bias=0.05 / no-opt-cu-delta-qp / no-aq-motion / no-hdr / no-hdr-opt / no-dhdr10-opt / no-idr-recovery-sei / analysis-reuse-level=5 / scale-factor=0 / refine-intra=0 / refine-inter=0 / refine-mv=1 / refine-ctu-distortion=0 / no-limit-sao / ctu-info=0 / no-lowpass-dct / refine-analysis-type=0 / copy-pic=1 / max-ausize-factor=1.0 / no-dynamic-refine / no-single-sei / no-hevc-aq / no-svt / no-field / qp-adaptation-range=1.00
Default                                  : Yes
Forced                                   : No
Color range                              : Limited
Color primaries                          : BT.601 PAL
Transfer characteristics                 : BT.709
Matrix coefficients                      : BT.601

Additional information

Actually everything work perfectly fine on my system, watching h264 1080p video with mpv or online with firefox and html5, playing high end 3d games with max settings using wine, never encountered any problem or crash.

Start by reproducing the issue with mpv 0.32.0, the reported Debian sid and AMD driver setup, and the hwdec=vaapi and hwdec=vdpau options. Inspect the attached VAAPI and VDPAU logs alongside the two sample descriptions; done means HEVC playback works through both hardware-decoding paths without green artifacts or a black screen.