Out of range float to integer conversion UB with fuzzed files

Version information

• mpv version: mpv v0.37.0-484-g665a472098
• Source of the mpv binary: build from source with UBSan enabled

Reproduction steps

1. Compile commit 665a472 (current master as of writing) with UBSan enabled: CC=clang meson setup build-ubsan -Dlibmpv=true -Db_sanitize=undefined
2. Play the sample files with this build

UB in demux/demux_mkv.c:1691 and demux/demux_mkv.c:1806:

user@laptop:~/fuzz-campaign/hits/samples$ ~/mpv/build-ubsan/mpv UB-demux_mkv
[mkv] Next subelement content goes past end of containing element, will be truncated
[mkv] Subelement headers go past end of containing element
[mkv] Next subelement content goes past end of containing element, will be truncated
[mkv] Next subelement content goes past end of containing element, will be truncated
[mkv] Next subelement content goes past end of containing element, will be truncated
[mkv] Error parsing element Tracks
[mkv] mkv metadata beyond end of file - incomplete file?
../demux/demux_mkv.c:1691:24: runtime error: -nan is outside the range of representable values of type 'unsigned int'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../demux/demux_mkv.c:1691:24 in 
../demux/demux_mkv.c:1806:51: runtime error: -nan is outside the range of representable values of type 'unsigned int'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../demux/demux_mkv.c:1806:51 in 
[mkv] Corrupt file detected. Trying to resync starting from position 9482...
[mkv] Cluster found at 12531.
[mkv] Invalid EBML length at position 13342
[mkv] Corrupt file detected. Trying to resync starting from position 13342...
 (+) Video --vid=1 (*) (mpeg4 640x480 23.976fps)
 (+) Audio --aid=1 --alang=eng (*) (aac 6ch 16418Hz)
     Audio --aid=2 --alang=jpn (aac 6ch)
 (+) Subs  --sid=1 --slang=eng (*) (subrip)
     Subs  --sid=2 --slang=eng (*) (dvd_subtitle)
[ffmpeg/video] mpeg4: Error at MB: 212
[ffmpeg/video] mpeg4: marker does not match f_code
    --------  8<  --------

UB in filters/f_output_chain.c:190, filters/f_output_chain.c:200 and common/common.c:51:

user@laptop:~/fuzz-campaign/hits/samples$ ~/mpv/build-ubsan/mpv UB-f_output_chain-common
[mkv] Next subelement content goes past end of containing element, will be truncated
[mkv] Subelement headers go past end of containing element
[mkv] Error parsing subelement id
[mkv] Next subelement content goes past end of containing element, will be truncated
[mkv] Error parsing element Tracks
[mkv] Invalid EBML length at position 5072
[mkv] mkv metadata beyond end of file - incomplete file?
 (+) Video --vid=1 (*) (mpeg4 640x480 23.976fps)
 (+) Audio --aid=1 --alang=eng (*) (aac 6ch)
     Audio --aid=2 --alang=jpn (aac 1ch 22050Hz)
[mkv] Invalid EBML length at position 16296
[mkv] Corrupt file detected. Trying to resync starting from position 16296...
[ffmpeg/audio] aac: noise_facs_q 254 is invalid
[ffmpeg/audio] aac: Reserved SBR extensions is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.
[ffmpeg/audio] aac: If you want to help, upload a sample of this file to https://streams.videolan.org/upload/ and contact the ffmpeg-devel mailing list. (ffmpeg-devel@ffmpeg.org)
[ffmpeg/audio] aac: Expected to read 17 SBR bytes actually read 29.
[ffmpeg/audio] aac: Reserved SBR extensions is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.
[ffmpeg/audio] aac: If you want to help, upload a sample of this file to https://streams.videolan.org/upload/ and contact the ffmpeg-devel mailing list. (ffmpeg-devel@ffmpeg.org)
[ffmpeg/audio] aac: Expected to read 17 SBR bytes actually read 29.
[ffmpeg/audio] aac: This stream seems to incorrectly report its last channel as SCE[1], mapping to LFE[0]
[ffmpeg/audio] aac: noise_facs_q 254 is invalid
../filters/f_output_chain.c:190:30: runtime error: -nan is outside the range of representable values of type 'long'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../filters/f_output_chain.c:190:30 in 
../filters/f_output_chain.c:200:31: runtime error: -nan is outside the range of representable values of type 'long'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../filters/f_output_chain.c:200:31 in 
[ffmpeg/video] mpeg4: ac-tex damaged at 3 5
[ffmpeg/video] mpeg4: Error at MB: 208
[ffmpeg/video] mpeg4: marker does not match f_code
[ffmpeg/video] mpeg4: marker does not match f_code
[ffmpeg/video] mpeg4: marker does not match f_code
[ffmpeg/video] mpeg4: marker does not match f_code
[ffmpeg/video] mpeg4: marker does not match f_code
[ffmpeg/video] mpeg4: marker does not match f_code
[ffmpeg/video] mpeg4: marker does not match f_code
[ffmpeg/video] mpeg4: marker does not match f_code
[ffmpeg/video] mpeg4: marker does not match f_code
[ffmpeg/video] mpeg4: marker does not match f_code
[ffmpeg/video] mpeg4: marker does not match f_code
[ffmpeg/video] mpeg4: ac-tex damaged at 0 0
[ffmpeg/video] mpeg4: Error at MB: 0
AO: [pipewire] 8000Hz 5.1 6ch floatp
VO: [gpu] 640x480 yuv420p
[ffmpeg/audio] aac: Number of bands (56) exceeds limit (39).
Error decoding audio.
[ffmpeg/audio] aac: Number of bands (46) exceeds limit (39).
Error decoding audio.
[ffmpeg/audio] aac: invalid band type
Error decoding audio.
[ffmpeg/audio] aac: Number of bands (65) exceeds limit (39).
Error decoding audio.
[ffmpeg/audio] aac: Number of bands (41) exceeds limit (39).
Error decoding audio.
[ffmpeg/audio] aac: invalid band type
Error decoding audio.
[ffmpeg/audio] aac: Number of bands (57) exceeds limit (39).
Error decoding audio.
../common/common.c:51:27: runtime error: inf is outside the range of representable values of type 'long long'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../common/common.c:51:27 in 
No video PTS! Making something up. Using 23.976000 FPS.
AV: 00:00:00 / -2562047788015215:-30:-8 (0%) A-V:  0.000
Exiting... (End of file)

UB in player/playloop.c:577:

user@laptop:~/fuzz-campaign/hits/samples$ ~/mpv/build-ubsan/mpv UB-playloop
[mkv] Error parsing subelement length
[mkv] Next subelement content goes past end of containing element, will be truncated
[mkv] Subelement headers go past end of containing element
[mkv] Next subelement content goes past end of containing element, will be truncated
[mkv] Next subelement content goes past end of containing element, will be truncated
[mkv] Next subelement content goes past end of containing element, will be truncated
[mkv] Subelement content goes past end of containing element
[mkv] Error parsing element Tracks
[mkv] Missing codec ID!
[mkv] mkv metadata beyond end of file - incomplete file?
 (+) Audio --aid=1 --alang=eng (*) (aac 6ch)
     Audio --aid=2 --alang=jpn (aac 6ch 44100Hz)
 (+) Subs  --sid=1 --slang=eng (*) (subrip)
     Subs  --sid=2 --slang=eng (dvd_subtitle)
[ffmpeg/audio] aac: noise_facs_q 254 is invalid
[ffmpeg/audio] aac: Reserved SBR extensions is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.
[ffmpeg/audio] aac: If you want to help, upload a sample of this file to https://streams.videolan.org/upload/ and contact the ffmpeg-devel mailing list. (ffmpeg-devel@ffmpeg.org)
[ffmpeg/audio] aac: Expected to read 17 SBR bytes actually read 29.
[ffmpeg/audio] aac: Reserved SBR extensions is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.
[ffmpeg/audio] aac: If you want to help, upload a sample of this file to https://streams.videolan.org/upload/ and contact the ffmpeg-devel mailing list. (ffmpeg-devel@ffmpeg.org)
[ffmpeg/audio] aac: Expected to read 17 SBR bytes actually read 29.
[ffmpeg/audio] aac: This stream seems to incorrectly report its last channel as SCE[1], mapping to LFE[0]
[ffmpeg/audio] aac: noise_facs_q 254 is invalid
../filters/f_output_chain.c:190:30: runtime error: -nan is outside the range of representable values of type 'long'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../filters/f_output_chain.c:190:30 in 
../filters/f_output_chain.c:200:31: runtime error: -nan is outside the range of representable values of type 'long'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../filters/f_output_chain.c:200:31 in 
AO: [pipewire] 8000Hz 5.1 6ch floatp
[ffmpeg/audio] aac: Number of bands (56) exceeds limit (39).
Error decoding audio.
[ffmpeg/audio] aac: channel element 3.13 is not allocated
Error decoding audio.
[ffmpeg/audio] aac: channel element 3.7 is not allocated
Error decoding audio.
[ffmpeg/audio] aac: channel element 3.7 is not allocated
Error decoding audio.
[ffmpeg/audio] aac: Sample rate index in program config element does not match the sample rate index configured by the container.
../common/common.c:51:27: runtime error: -nan is outside the range of representable values of type 'long long'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../common/common.c:51:27 in 
../player/playloop.c:577:27: runtime error: -nan is outside the range of representable values of type 'int'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../player/playloop.c:577:27 in 
AO: [pipewire] 8000Hz unknown16 (empty) 16ch floatp
[ffmpeg/audio] aac: Number of bands (52) exceeds limit (39).
Error decoding audio.
[ffmpeg/audio] aac: skip_data_stream_element: Input buffer exhausted before END element found
Error decoding audio.
[ffmpeg/audio] aac: SBR was found before the first channel element.
[ffmpeg/audio] aac: Reserved bit set.
[ffmpeg/audio] aac: Number of bands (7) exceeds limit (5).
Error decoding audio.
[mkv] Corrupt file detected. Trying to resync starting from position 12413...
A: -2562047788015215:-30:-8 / -2562047788015215:-30:-8
Exiting... (End of file)

Expected behavior

It would be best if there no float to integer conversion undefined behaviors in the code.

I don't know the project's attitude towards these low impact UBs. Practically speaking these issues probably have little impact unless the compilers decide to do something really crazy in the future. Regardless, guarding these float to int conversion points with some sensible clamp is a good idea imo.

Sample files

https://0x0.st/H7xe.zip

Start with the reported conversions in demux/demux_mkv.c, filters/f_output_chain.c, common/common.c, and player/playloop.c. Build mpv with the shown UBSan Meson options and run the linked sample files to reproduce the diagnostics. Done means the listed float-to-integer conversions no longer trigger UBSan errors for these samples.