swiftlang / swiftlang/swift-corelibs-libdispatch

[SR-2905] Linux libpwq: Dispatch queues + blocking tasks = poor performance

Open
#719 7 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug libdispatch
Dominant language
C
Stars
2.6k
Forks
496
Avg merge
2d 6h
Merged PRs (30d)
3

Description

Previous ID SR-2905
Radar None
Original Reporter fumoboy007 (JIRA User)
Type Bug
Environment

Linux, Swift 3.0

Additional Detail from JIRA
Votes 1
Component/s libdispatch
Labels Bug
Assignee dgrove-oss (JIRA)
Priority Medium

md5: ef7acc516d0911e996748b91cd397e7a

Issue Description:

Consider this code:

for i in 1...100 {
        DispatchQueue.global().async {
                print(i)
                sleep(1000)
        }
}
dispatchMain()

Environment: Linux, 8 cores

Expected outcome: The first 8 work items start running. When the threads block on the sleep system call, the manager creates additional threads so that the number of active threads is always 8.

Actual outcome: The first 8 work items start running. Then every 1 second, one new work item runs. It takes 93 seconds for all the print statements to run.

There are actually two issues here.

The first issue is, on every run of the manager, the manager only creates 1 additional worker thread even though all of the existing threads are blocked. Instead, it should create min(num_blocked_threads, cpu_count) worker threads so that we always have cpu_count active threads.

The second issue is the manager only runs once a second. I assume this is to prevent the manager thread from taking away significant CPU time from other threads. Is it possible to achieve the same result by using thread priorities? In other words, can we set the manager thread priority to a special number such that the thread scheduler will only schedule it when the CPU is idle?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Run the supplied DispatchQueue.global async/sleep reproduction on Linux and compare its timing and printed output with the stated expected behavior. No file or test is named; inspect the libdispatch worker-thread manager and its scheduling interval. Done means the reproduction no longer exhibits the reported delayed dispatch and the two manager concerns have clear, tested behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, linux, swift
Domain
operating-systems, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.