php / php/php-src

set_timeout() fails to terminate execution when IO Latency is high

Open
#11,971 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Feature Status: Needs Triage
Dominant language
C
Stars
40.4k
Forks
8.1k
Avg merge
2d 13h
Merged PRs (30d)
96

Description

Description

I'm not sure if this is a bug or not and I haven't found the proper channel to ask (shall I raise a discussion in Internals?).

We faced a problem in our production cluster. Our setup is following:

SAPI: fpm
max_execution_time: 28
hard_timeout: 2

When Opcache is reloading, we're having excessive IO pressure, which is expected - PHP starts reading everything from disk, etc... However, scripts started executing for 5+ minutes - that wasn't expected.

Further investigation showed that zend_set_timeout_ex uses setitimer(ITIMER_PROF) by default, which means the time the process spends in IOWait doesn't seem to count.

So, basically, even though our gateway cannot wait for such long - it expects that application will finish earlier, PHP still performs the request. One more backside of such behaviour - the restart of Opcache takes longer - it waits until all the workers finish execution (or become killed by opcache when opcache.force_restart_timeout is reached).

This problem doesn't seem to appear when we use ITIMER_REAL. I can provide how we tested this.

I can see a couple of solutions here:

  • Switch to ITIMER_REAL (it looks like there are some concerns about the Apache module)
  • Choose one over another for some SAPIs when we are sure that there shouldn't be a conflict (e.g. FPM, CLI)
  • Make this behaviour configurable (either at runtime or compile time)
  • Something else?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at zend_set_timeout_ex and compare the reported behavior of ITIMER_PROF with ITIMER_REAL under FPM and high I/O latency. Review the Apache-module concern and the proposed alternatives; done requires a decided timer behavior or configuration approach that prevents requests from running for several minutes.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, php
Domain
operating-systems
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.