P3 - Nvidia decoding sometimes returns CUDA_ERROR_UNKNOWN
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 283
- Forks
- 72
- Avg merge
- 27m
- Merged PRs (30d)
- 1
Description
debug CUDA_ERROR_UNKNOWN errors Why? Should follow up, but hard to debug until P2s are addressed and seem to have stopped.
**Describe the bug**
The GPU video decoding fails with `CUDA_ERROR_UNKNOWN`, needing the user to restart the node for future segments. Sometimes it's paired with `CUDA_ERROR_OUT_OF_MEMORY` or `CUDA_ERROR_ILLEGAL_ADDRESS`.
**To Reproduce**
Steps to reproduce the behavior:
* Unclear as of now.
**Expected behavior**
Decrease the blast radius of these errors if possible, and figure out the root cause.
**Screenshots**
ERROR_UNKNOWN
ERROR_ILLEGAL_ADDRESS
ERROR_OUT_OF_MEMORY
**Additional context**
Stack-trace for future reference:
LPMS - https://github.com/livepeer/lpms/blob/master/ffmpeg/decoder.c#L250
FFmpeg - entry-point https://github.com/FFmpeg/FFmpeg/blob/870bfe16a12bf09dca3a4ae27ef6f81a2de80c40/libavutil/hwcontext.c#L610
most-probable line causing the error https://github.com/FFmpeg/FFmpeg/blob/870bfe16a12bf09dca3a4ae27ef6f81a2de80c40/libavutil/hwcontext.c#L629
cuda-specific ctx creation routine
https://github.com/FFmpeg/FFmpeg/blob/870bfe16a12bf09dca3a4ae27ef6f81a2de80c40/libavutil/hwcontext_cuda.c#L379
cuCtxCreate call https://github.com/FFmpeg/FFmpeg/blob/870bfe16a12bf09dca3a4ae27ef6f81a2de80c40/libavutil/hwcontext_cuda.c#L363
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at ffmpeg/decoder.c:250 and trace the failure through FFmpeg's libavutil/hwcontext.c entry point at line 610. Read the CUDA context creation path in libavutil/hwcontext_cuda.c around cuCtxCreate at lines 363-379, then investigate reproducible CUDA_ERROR_UNKNOWN, CUDA_ERROR_OUT_OF_MEMORY, and CUDA_ERROR_ILLEGAL_ADDRESS cases. Done means the root cause is identified and the decoder's failure impact is reduced or documented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- c
- Domain
- audio-video-rtc
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100