google / google/oss-fuzz

Minimizing timeout test cases

Open
#11,950 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Shell
Stars
12.6k
Forks
2.9k
Avg merge
2d 2h
Merged PRs (30d)
62

Description

In my experience, test cases for timeouts are not minimized. I'm not sure if this is intentional, or a failure of the system to do the minimization. When triggering minimization manually, I get "Minimize task errored out: LibFuzzer minimization failed.". Given how painful it is to minimize these locally, it would be very useful to support this, including setting a reasonable timeout value.

In many cases, timeouts relate directly to the size of the input. In igraph, we encountered cases where it's the specific structure of the input was to blame. Obtaining a small test case is usually necessary to debug these.

Contributor guide

Open the contributing guide

Research direction

Start by tracing the manual minimization path described in the issue and the LibFuzzer failure message. Determine how timeout cases are currently handled, where a minimization timeout would be configured, and what successful minimization of a timeout should produce; the issue names no files or tests.

Written by the indexing model from the issue text.

Assessment

Domain
testing-qa
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.