set_timeout() fails to terminate execution when IO Latency is high
Nobody has claimed this yet.
- Dominant language
- C
- Stars
- 40.4k
- Forks
- 8.1k
- Avg merge
- 2d 13h
- Merged PRs (30d)
- 96
Description
Description
I'm not sure if this is a bug or not and I haven't found the proper channel to ask (shall I raise a discussion in Internals?).
We faced a problem in our production cluster. Our setup is following:
SAPI: fpm
max_execution_time: 28
hard_timeout: 2
When Opcache is reloading, we're having excessive IO pressure, which is expected - PHP starts reading everything from disk, etc... However, scripts started executing for 5+ minutes - that wasn't expected.
Further investigation showed that zend_set_timeout_ex uses setitimer(ITIMER_PROF) by default, which means the time the process spends in IOWait doesn't seem to count.
So, basically, even though our gateway cannot wait for such long - it expects that application will finish earlier, PHP still performs the request. One more backside of such behaviour - the restart of Opcache takes longer - it waits until all the workers finish execution (or become killed by opcache when opcache.force_restart_timeout is reached).
This problem doesn't seem to appear when we use ITIMER_REAL. I can provide how we tested this.
I can see a couple of solutions here:
- Switch to
ITIMER_REAL(it looks like there are some concerns about the Apache module) - Choose one over another for some SAPIs when we are sure that there shouldn't be a conflict (e.g. FPM, CLI)
- Make this behaviour configurable (either at runtime or compile time)
- Something else?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at zend_set_timeout_ex and compare the reported behavior of ITIMER_PROF with ITIMER_REAL under FPM and high I/O latency. Review the Apache-module concern and the proposed alternatives; done requires a decided timer behavior or configuration approach that prevents requests from running for several minutes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- c, php
- Domain
- operating-systems
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100