intel / intel/gprofiler

Flaky phpspy tests in when profiling host PHP

Open
#630 0 comments 0 reactions 0 assignees View on GitHub
bug runtime/php tests
Dominant language
Python
Stars
826
Forks
79
PR merge metrics
No merged PRs in 30d

Description

Example run: https://github.com/Granulate/gprofiler/actions/runs/3696159264/jobs/6261770730, https://github.com/Granulate/gprofiler/actions/runs/3703253415/jobs/6274861085

Seeing errors in the phpspy stderr, but it's hard to correlate:
```
[2022-12-14 17:21:29,362] DEBUG: gprofiler.profilers.php: phpspy stderr (phpspy_stderr=awk: cannot open /proc/3516/maps (No such file or directory)
get_php_bin_path: Failed
error: pgrep mode: main_pid routine returned non ok status and PID (3516) is not dead, rv = 1, errno - 0
objdump: /proc/3497/root//tmp/_MEIiqKcwW/gprofiler/resources/php/phpspy: not a dynamic object
get_symbol_offset: Failed
error: pgrep mode: main_pid routine returned non ok status and PID (3497) is not dead, rv = 1, errno - 0)
[2022-12-14 17:21:29,363] INFO: gprofiler.profilers.perf: Starting perf (fp mode)
[2022-12-14 17:21:29,364] DEBUG: gprofiler.utils: Running command: (/tmp/_MEIiqKcwW/gprofiler/resources/perf record -F 11 -a -g -o /tmp/gprofiler_tmp/tmp52nq1qnj/perf.fp --switch-output=signal -m 129)
[2022-12-14 17:21:29,477] INFO: gprofiler.profilers.perf: Started perf (fp mode)
[2022-12-14 17:21:29,477] INFO: gprofiler.profilers.perf: Starting perf (dwarf mode)
[2022-12-14 17:21:29,478] DEBUG: gprofiler.utils: Running command: (/tmp/_MEIiqKcwW/gprofiler/resources/perf record -F 11 -a -g -o /tmp/gprofiler_tmp/tmp52nq1qnj/perf.dwarf --switch-output=signal -m 257 --call-graph dwarf,8192)
[2022-12-14 17:21:29,597] INFO: gprofiler.profilers.perf: Started perf (dwarf mode)
[2022-12-14 17:21:32,611] DEBUG: gprofiler.profilers.php: phpspy stderr (phpspy_stderr=get_symbol_offset: Failed
get_symbol_offset: Failed
get_symbol_offset: Failed
get_symbol_offset: Failed
get_symbol_offset: Failed
try_get_php_version: Unrecognized PHP version
error: pgrep mode: main_pid routine returned non ok status and PID (911) is not dead, rv = 1, errno - 0
try_get_php_version: Unrecognized PHP version
error: pgrep mode: main_pid routine returned non ok status and PID (915) is not dead, rv = 1, errno - 0
try_get_php_version: Unrecognized PHP version
error: pgrep mode: main_pid routine returned non ok status and PID (669) is not dead, rv = 1, errno - 0
try_get_php_version: Unrecognized PHP version
error: pgrep mode: main_pid routine returned non ok status and PID (3294) is not dead, rv = 1, errno - 0
awk: cannot open /proc/3836/maps (No such file or directory)
get_php_bin_path: Failed
error: pgrep mode: main_pid routine returned non ok status and PID (3836) is not dead, rv = 1, errno - 0
objdump: /proc/3497/root//tmp/_MEIiqKcwW/gprofiler/resources/php/phpspy: not a dynamic object
get_symbol_offset: Failed
get_symbol_offset: Failed
get_symbol_offset: Failed
try_get_php_version: Unrecognized PHP version
error: pgrep mode: main_pid routine returned non ok status and PID (3294) is not dead, rv = 1, errno - 0
try_get_php_version: Unrecognized PHP version
error: pgrep mode: main_pid routine returned non ok status and PID (911) is not dead, rv = 1, errno - 0
try_get_php_version: Unrecognized PHP version
error: pgrep mode: main_pid routine returned non ok status and PID (669) is not dead, rv = 1, errno - 0
try_get_php_version: Unrecognized PHP version
error: pgrep mode: main_pid routine returned non ok status and PID (915) is not dead, rv = 1, errno - 0)
```

The test is flaky and passes sometimes while failing other times. I am currently marking it as "skip".

This was found during https://github.com/Granulate/gprofiler/pull/621 and during that time, GH have changed the `ubuntu-latest` runner to be 22.04. I tried reverting to 20.04 (which was the latest beforehand) but it didn't help. I still bet it's something about the versions.

EDIT: in https://github.com/Granulate/gprofiler/pull/396/commits/f01adc56f856ff68536d3cab2c824cabe5427e1f I have correlated the problem to profiling PHP on the host, it seems that profling a PHP container works fine, but when it's host then possibly the version we get from apt differs, or was upgraded and is now flaky. Currently, all phpspy tests with `in_container = False` are skipped.

Contributor guide

Open the contributing guide

Research direction

Start with the skipped phpspy tests where in_container=False and reproduce them in the GitHub Actions host environment. Compare the host and container PHP versions and the phpspy stderr across runner versions. Done means the host profiling tests run reliably or the failure condition is clearly isolated and covered by the test setup.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions, php, python
Domain
ci-cd, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.