pybind / pybind/pybind11

[QUESTION]code stops on condition_variable.wait

Open
#3,032 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
18k
Forks
2.3k
Avg merge
5d 17h
Merged PRs (30d)
10

Description

I am trying to implement a thread-safe queue in c++ with std::queue, std::mutex and std::contion_variable. I want to use pybind11 to call it from python. The whole logic is a multiple producers and multiple consumers.
I started consumers first, but found the whole progress was blocked at line 4 in C++(from line 12 in PYTHON), so the producers would not start and it will always be blocked here. I also tried ThreadSafeQueue in c++, the logic is same as python and the code acted normally: producers started, consumers were waken.

is this blocking in python normal? and why? Thanks for reply.

**C++:**
1 int ThreadSafeQueue::wait_pop() {
2    std::unique_lock<std::mutex> lock(_mutex);
3    while (_queue.empty()) {
4        _condition_variable.wait(lock);
5    }
6    int value = _queue.front();
7    _queue.pop();
8    ++_consumer_count;
9    return value;
10 }

**PYTHON:**
1 q = thread_safe_queue.ThreadSafeQueue()
2
3 # multi consumers
4 def consume_func():
5     for i in range(CONSUME_NUM_PER_THREAD):
6         poped_val = q.wait_pop()
7
8 consumer_threads = []
9 for i in range(CONSUMER_NUM):
10    t = threading.Thread(target=consume_func)
11    consumer_threads.append(t)
12    t.start()
13
14 ... ... # producers logic would not start

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with ThreadSafeQueue::wait_pop and the Python consume_func shown in the issue, then reproduce the case with consumers started before producers. Compare the condition_variable wait path with the producer startup and pybind11 threading behavior; done means explaining whether the blocking is expected and identifying the cause.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, python
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.