LizardByte / LizardByte/Sunshine

Sunshine crashes (nvcuda64.dll, c0000005) when creating 10-bit HEVC NVENC encoder on RTX 4060

Open
#5,580 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
41.3k
Forks
2.1k
Avg merge
23h 47m
Merged PRs (30d)
124

Description

### Is there an existing issue for this?

- [x] I have searched the existing issues

### Is your issue described in the documentation?

- [x] I have read the documentation

### Is your issue present in the latest beta/pre-release?

None

### Describe the Bug

**Describe the bug**
Sunshine crashes with an access violation (c0000005) in nvcuda64.dll every time it tries to create the second-pass HEVC 10-bit NVENC encoder during the encoder testing phase. This happens consistently, even when running sunshine.exe manually (not as a service).

**To Reproduce**
1. Run sunshine.exe (manually via cmd, or via service)
2. Sunshine successfully creates: H.264 (8-bit), HEVC (8-bit), AV1 (8-bit), H.264 yuv444
3. Crashes immediately when creating HEVC 10-bit encoder (Color coding: SDR Rec. 709, Color depth: 10-bit)

**Expected behavior**
Sunshine should either successfully create the 10-bit encoder, or fail gracefully and fall back to another encoder, instead of crashing the entire process.

**System info**
- GPU: NVIDIA GeForce RTX 4060 (Desktop)
- OS: Windows 10 Pro 22H2
- Sunshine versions tested: v2025.628.4510 and v2025.924.154138 (both crash identically)
- NVIDIA drivers tested: 616.56, 591.74 (both crash identically, ruling out driver issue)
- Also tried: full DDU clean uninstall + reinstall, still crashes
- Workaround: setting `encoder = software` in sunshine.conf avoids the crash entirely

**Crash details (from Windows Event Viewer, Application Error)**
- Faulting module: nvcuda64.dll
- Exception code: 0xc0000005
- Consistent across all 3 driver versions tested (32.0.16.1656, 32.0.15.9597, 32.0.15.9174)

**Logs**
[Adjunta aquí una captura o copia del log de consola desde "Trying encoder [nvenc]" hasta el crash]

### Expected Behavior

_No response_

### Additional Context

_No response_

### Host Operating System

Windows

### Operating System Version

win 10

### Architecture

amd64/x86_64

### Package

Windows - exe installer

### GPU Type

NVIDIA

### GPU Model

Geforce rtx 4060

### GPU Driver/Mesa Version

591.74 (also tested with 616.56)

### Capture Method

Desktop Duplication API (Windows)

### Apps

```json

```

### Log output

```shell
[2026-08-30 16:09:16.720]: Info: // Testing for available encoders, this may generate errors. You can safely ignore those errors. //
[2026-08-30 16:09:16.720]: Info: Trying encoder [nvenc]
[2026-08-30 16:09:16.968]: Info:
Device Description : NVIDIA GeForce RTX 4060
Device Vendor ID : 0x000010DE
Device Device ID : 0x00002882
Device Video Mem : 7957 MiB
Device Sys Mem : 0 MiB
Share Sys Mem : 16356 MiB
Feature Level : 0x0000B100
Capture size : 1920x1080
Offset : 0x0
Virtual Desktop : 1920x1080
[2026-08-30 16:09:16.970]: Info: Active GPU has HAGS disabled
[2026-08-30 16:09:16.970]: Info: Using realtime GPU priority
[2026-08-30 16:09:16.973]: Info: Creating encoder [h264_nvenc]
[2026-08-30 16:09:16.973]: Info: Color coding: SDR (Rec. 601)
[2026-08-30 16:09:16.974]: Info: Color depth: 8-bit
[2026-08-30 16:09:16.974]: Info: Color range: JPEG
[2026-08-30 16:09:17.046]: Info: NvEnc: created encoder H.264 P1 async two-pass rfi
[2026-08-30 16:09:17.074]: Info: Creating encoder [hevc_nvenc]
[2026-08-30 16:09:17.074]: Info: Color coding: SDR (Rec. 601)
[2026-08-30 16:09:17.074]: Info: Color depth: 8-bit
[2026-08-30 16:09:17.076]: Info: Color range: JPEG
[2026-08-30 16:09:17.140]: Info: NvEnc: created encoder HEVC P1 async two-pass rfi
[2026-08-30 16:09:17.168]: Info: Creating encoder [av1_nvenc]
[2026-08-30 16:09:17.168]: Info: Color coding: SDR (Rec. 601)
[2026-08-30 16:09:17.168]: Info: Color depth: 8-bit
[2026-08-30 16:09:17.169]: Info: Color range: JPEG
[2026-08-30 16:09:17.222]: Info: NvEnc: created encoder AV1 P1 async two-pass rfi
[2026-08-30 16:09:17.246]: Info: Creating encoder [h264_nvenc]
[2026-08-30 16:09:17.246]: Info: Color coding: SDR (Rec. 601)
[2026-08-30 16:09:17.247]: Info: Color depth: 8-bit
[2026-08-30 16:09:17.248]: Info: Color range: JPEG
[2026-08-30 16:09:17.303]: Info: NvEnc: created encoder H.264 P1 async yuv444 two-pass rfi
[2026-08-30 16:09:17.550]: Info:
Device Description : NVIDIA GeForce RTX 4060
Device Vendor ID : 0x000010DE
Device Device ID : 0x00002882
Device Video Mem : 7957 MiB
Device Sys Mem : 0 MiB
Share Sys Mem : 16356 MiB
Feature Level : 0x0000B100
Capture size : 1920x1080
Offset : 0x0
Virtual Desktop : 1920x1080
[2026-08-30 16:09:17.555]: Info: Active GPU has HAGS disabled
[2026-08-30 16:09:17.555]: Info: Using realtime GPU priority
[2026-08-30 16:09:17.557]: Info: Creating encoder [hevc_nvenc]
[2026-08-30 16:09:17.557]: Info: Color coding: SDR (Rec. 709)
[2026-08-30 16:09:17.558]: Info: Color depth: 10-bit
[2026-08-30 16:09:17.558]: Info: Color range: JPEG

--- PROCESS CRASHES HERE, no further log output ---

Windows Event Viewer (Application Error) details:
Faulting application name: sunshine.exe
Faulting module name: nvcuda64.dll, version: 32.0.15.9174
Exception code: 0xc0000005
Faulting module path: C:\WINDOWS\system32\DriverStore\FileRepository\nv_dispi.inf_amd64_7840b4313191ae17\nvcuda64.dll
```

### Online logs

_No response_

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the encoder-testing phase from sunshine.exe with the reported RTX 4060 and inspect the NVENC path reached after “Creating encoder [hevc_nvenc]” for SDR Rec. 709 10-bit output. Compare it with the preceding H.264, HEVC, AV1, and yuv444 encoder attempts. Done means the HEVC 10-bit test succeeds or fails gracefully without crashing the process, with fallback behavior preserved.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
audio-video-rtc, backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.