osquery / osquery/osquery

Fix flaky test ProcessesTest.test_sanity

Open
#8,130 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

flaky test macOS
Dominant language
C++
Stars
23.6k
Forks
2.6k
Avg merge
6d 7h
Merged PRs (30d)
14

Description

80: [----------] 1 test from ProcessesTest
80: [ RUN      ] ProcessesTest.test_sanity
80: /Users/runner/work/osquery/osquery/workspace/src/tests/integration/tables/helper.cpp:173: Failure
80: Value of: boost::get<CustomCheckerType>(validator)(value)
80:   Actual: false
80: Expected: true
80: Custom validator of the column "start_time" with value "1693090556" failed
80: Row: {cmdline: "/System/Library/CoreServices/loginwindow.app/Contents/MacOS/loginwindow console", cpu_subtype: "3", cpu_type: "16777223", cwd: "/", disk_bytes_read: "11366400", disk_bytes_written: "114688", egid: "20", euid: "501", gid: "20", name: "loginwindow", nice: "0", on_disk: "1", parent: "1", path: "/System/Library/CoreServices/loginwindow.app/Contents/MacOS/loginwindow", pgroup: "140", pid: "140", resident_size: "21041152", root: "", sgid: "20", start_time: "1693090556", state: "R", suid: "0", system_time: "342", threads: "2", total_size: "8765440", translated: "0", uid: "0", upid: "140", uppid: "1", user_time: "244", wired_size: "0"}
80: [  FAILED  ] ProcessesTest.test_sanity (84 ms)
80: [----------] 1 test from ProcessesTest (84 ms total)

After fixing and stabilizing how the process start time is being taken on macOS, this test started being flaky.
Looking at the test code:

https://github.com/osquery/osquery/blob/8cd1437805d14a0564a68603fcf5fac18fe3148f/tests/integration/tables/processes.cpp#L35-L53

There could be multiple reasons, starting from the fact that to get the boot time we are subtracting the current time with the uptime, which have to be taken in two slightly different moments.
Furthermore the function getUptime already returns the difference from the current time and the boot time, on macOS.

I think it would make sense to add a print to see what's the calculated boot time and current time, and maybe add a direct function to get the boot time, instead of passing through getUptime.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with tests/integration/tables/processes.cpp lines 35-53 and the failing ProcessesTest.test_sanity output. Inspect how getUptime is used to calculate boot time on macOS, comparing the current-time and uptime reads. Done means the process start-time validation is stable and the flaky test passes consistently.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
operating-systems, testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.