NVIDIA / NVIDIA/stdexec

exec::split and exec::ensure_started bypass domain customization and hardcode std::mutex, causing deadlocks in cooperative schedulers

Open
#2,110 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
2.4k
Forks
270
Avg merge
3d 6h
Merged PRs (30d)
39

Description

Description

While integrating stdexec into the HPX project's P2300 infrastructure, we discovered that exec::split and exec::ensure_started cannot be customized via Execution Domains to use cooperative synchronization primitives. This results in deadlocks when running on user-level threading frameworks.

The Problem

There are two interconnected issues preventing us from adapting split for HPX:

1. Hardcoded OS-level blocking
In exec/detail/shared.hpp, the __shared::__sndr state utilizes a hardcoded std::mutex. For user-level threading frameworks like HPX, blocking the OS thread via std::mutex starves the worker pool. If a split sender completes on an HPX worker thread and a waiter is blocked on that same thread (e.g., when running with a single OS thread pool), it causes an immediate deadlock. We need to use our cooperative primitives (hpx::spinlock and hpx::condition_variable_any).

2. Bypassing Domain Customization
We attempted to fix this downstream by intercepting split_t and ensure_started_t inside our HPX execution domain (transform_sender) to route them to a custom shared state.
However, exec::split_t implements transform_sender as a static member of the CPO itself:

// In exec/split.hpp
struct split_t {
    template <class _CvSender>
    static constexpr auto transform_sender(set_value_t, _CvSender&& __sndr, __ignore) {
        return __shared::__sndr{split_t(), ...};
    }
};

Because stdexec resolves this static member before consulting the domain, our domain's transform_sender never gets the opportunity to intercept the lazy sender. It is eagerly converted into the std::mutex-backed __shared::__sndr.

Proposed Solutions
To support HPC and cooperative scheduling environments, we need a way to override this behavior. Some potential directions:

Defer to Domains: Alter the resolution order or remove the static transform_sender from the CPO so that custom domains can intercept
split_t before it resolves to __shared::__sndr.

Pluggable Synchronization: Allow the synchronization primitives inside __shared::__state to be injected via an environment query, allocator, or template policy, rather than hardcoding std::mutex.

Is there currently a recommended workaround for this, or would the team be open to a PR addressing this architectural limitation?

Found this issue while Implementing Senders and receivers in hpx

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading exec/split.hpp and exec/detail/shared.hpp, focusing on split_t, ensure_started_t, __shared::__sndr, and the __shared::__state synchronization. Trace how domain transform_sender resolution reaches the shared state and identify existing customization or testing entry points. Done means cooperative schedulers can customize these operations without being forced through the hardcoded std::mutex path.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
backend-api-design, operating-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.