obsproject / obsproject/obs-studio

Crash (NULL deref) in screencast-portal.c start() when a delayed portal SelectSources response arrives after the session is gone

Open Beginner friendly
#13,809 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C
Stars
76.4k
Forks
10.2k
Avg merge
4d 23h
Merged PRs (30d)
12

Description

Operating system and version

Arch Linux (Omarchy), kernel 7.1.8-arch1-3, Wayland (Hyprland),
xdg-desktop-portal 1.20.x + xdg-desktop-portal-hyprland 1.4.1, glib2 2.88.3, NVIDIA RTX 4090 (610.57.04)

OBS Studio version

32.2.1 (Arch extra 32.2.1-7). The unguarded call is still present in current master.

Expected behavior

If the ScreenCast portal responds to SelectSources after the capture's session has been
torn down (e.g. the response arrives after the 25s D-Bus timeout already failed the request),
OBS should ignore the stale response.

Actual behavior

SIGSEGV (SEGV_MAPERR) in the main thread, 4 identical crashes in one session:

#0 g_variant_builder_add_value (value=0x0)     libglib-2.0.so.0
#1 ...                                          libglib-2.0.so.0
#2 g_variant_new_va                             libglib-2.0.so.0
#3 g_variant_new                                libglib-2.0.so.0
#4 <start()>                                    linux-pipewire.so + 0x165af
#5 <on_select_sources_response_received_cb>     linux-pipewire.so + 0x15c03
#6 emit_signal_instance_in_idle_cb              libgio-2.0.so.0
...

start() in plugins/linux-pipewire/screencast-portal.c calls
g_variant_new("(osa{sv})", capture->session_handle, "", &builder) with no NULL check on
capture->session_handle. OBS's own log ends at [pipewire] Asking for monitor and window
— the log line at the top of start() — microseconds before the crash.

Trigger / reproduction sketch

  1. Heavy capture renegotiation (multiple screen-capture sources created/destroyed, scene
    switches) on Hyprland; xdg-desktop-portal-hyprland becomes wedged under load
    (DMA-BUF alloc failures → SHM fallback → "Out of buffers", then unresponsive).
  2. OBS logs several Error creating screencast session: Timeout was reached (25s D-Bus timeout).
  3. When the portal recovers (~78s later in our capture), it emits the response to the stale
    SelectSources request. The GDBus signal subscription is still connected, but the capture's
    session_handle is NULL → crash in g_variant_new.

Suggested fix

NULL-check capture->session_handle at the top of start() (and defensively in
on_create_session_response_received_cb), logging and aborting the stale start instead of
crashing.

Evidence available

Four coredumps (systemd-coredump, byte-identical stacks at 12:10/12:36/12:40/12:55 local),
full OBS logs, and the portal-side journal showing the wedge and recovery timeline.
Happy to attach any of these.

Related but distinct: #5468, #11156, #9733 (other pipewire crashes; none match this
NULL-session-after-timeout path).

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Read plugins/linux-pipewire/screencast-portal.c, starting with start() and on_select_sources_response_received_cb; trace how session_handle can become NULL after the D-Bus timeout. Confirm that a delayed SelectSources response is ignored without crashing, while normal screen-capture startup still proceeds.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, linux
Domain
desktop
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
78/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.