compio-rs / compio-rs/compio

Poll driver sometimes stuck at 100% CPU utilization in the schedule function

Open
#984 7 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
1.9k
Forks
133
Avg merge
1d 4h
Merged PRs (30d)
20

Description

I'm running compio using the polling driver on some old 4.18.X kernel. Sometimes I saw some thread gets stuck in a bad state - it uses 100% CPU all the time, and taking a perf profile shows that majority of the CPU cycle is in the runtime scheduling function.

The workload is roughly the tcp service threads runs a TCP server and spawns work to the source client threads via a flume channel, and it seems to be related to a spike of number of request sent via the cross thread channel.

I'm running compio commit 1341f7253133cd6d9fabd8c6ed0660eedc1120f5, so maybe it's remediated by https://github.com/compio-rs/compio/pull/976?

```
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
1233339 root 20 0 8221.4g 27.5g 25.7g R 99.9 1.4 9:35.40 source-tcp-clie
1233340 root 20 0 8221.4g 27.5g 25.7g R 99.9 1.4 9:35.83 source-tcp-clie
1233337 root 20 0 8221.4g 27.5g 25.7g R 99.7 1.4 9:34.02 source-tcp-clie
1233338 root 20 0 8221.4g 27.5g 25.7g R 99.7 1.4 9:33.98 source-tcp-clie
1233489 root 20 0 8221.4g 27.5g 25.7g R 99.3 1.4 9:39.52 cave-tcp-servic
1233519 root 20 0 8221.4g 27.5g 25.7g R 99.3 1.4 9:40.27 cave-tcp-servic
```

Image

Contributor guide

Open the contributing guide

Research direction

Start with the polling driver and runtime scheduling path, then compare the behavior with the changes in pull request 976. Reproduce the high-CPU condition on an older 4.18.x kernel while generating a spike of cross-thread channel requests, and use perf to confirm where the loop remains busy. Done means the scheduler no longer spins at 100% CPU under this workload and the behavior is covered by a regression check.

Written by the indexing model from the issue text.

Assessment

Tech stack
linux, rust
Domain
operating-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.