PQ is not properly tonemapped to lower max brightness of HDR screens with vo=gpu-next

Important Information

Provide following Information:

• mpv version: mpv-dev-x86_64-v3-20240218-git-bd5b80b
• Platform and Version: Windows 11
• Source of the mpv binary: https://sourceforge.net/projects/mpv-player-windows/files/
• If known which version of mpv introduced the problem: was there all the time, going back two years at least.
• GPU model, driver and version: RTX 3000, Nvidia driver 551.61 but all back to 472 have been tested
• Possible screenshot or video of visual glitches, are attached

Reproduction steps

Setup the config file:
vo=gpu-next
target-colorspace-hint=yes
target-prim=bt.2020
target-trc=pq
hdr-compute-peak=no
target-peak=500

For target-peak use the value matching to your HDR display. It must be a HDR display!

Play back the attached calibration clip: ColorSteps_BT.2100_PQ.mkv

Expected behavior

While playing the attached calibration clip, the brightness of each color strip should increase up to the 235 value and not increase any further above that. Look at the attached expectation.png for reference.

Actual behavior

Look at the attached actual_photo.jpg, which was taken with an external device. The color field where the mouse cursor is, should not have been different from the two color fields left and right.
I also attached two screenshots: One was taken in jpg where this issue is visible and the other one was taking in avif where, interestingly, the issue is not visible.

Log file

is attached.

Sample files

directly attached are: expectation.png, actual_screenshot.jpg, log.txt
attached in the zip are: actual_screenshot.avif and ColorSteps_BT.2100_PQ.mkv

expectation.png:

actual_photo.jpg:

actual_screenshot.jpg:

additionalFiles.zip
log.txt

Reproduce the issue with vo=gpu-next using the listed config, ColorSteps_BT.2100_PQ.mkv, and attached log.txt on an HDR display set to the display's target peak. Compare the PQ ramp against expectation.png and the JPG/AVIF results. Done means values through 235 are tonemapped consistently to the configured lower peak without the reported brightness discontinuity.