google / google/oss-fuzz

md4c: Discrepancy between issues and crash stats.

Open
#11,543 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Shell
Stars
12.6k
Forks
2.9k
Avg merge
2d 2h
Merged PRs (30d)
62

Description

Project MD4C has one open problematic issue, namely https://oss-fuzz.com/testcase-detail/5609076764573696. It's not a crash but timeout, and it's marked as flaky. Even though I believe this bug is fixed for a long time, whenever any new (and unrelated) timeout issue gets detected, this issue behaves as a bucket of sorts and gets re-opened, even for fully reproducible testcases.

I understand this may be due technical limitations in classification of timeout issues (even though I'd wish that reproducible timeouts would not be bucketed with an issue marked as flaky) and at least in the past oss-fuzz interface allowed me to see the other/newer testcases grouped into this one by clicking on an icon in the list of all opened testcases and access the fresher ones that way.

However at this time the issue truly puzzles me: The group icon for accessing other test-cases bucketed into the same issue is not there and the report itself still declares it was last reproduced with the MD4C commit [c058e82c6a939f591059672dab08ddd0d4ce416e](https://github.com/mity/md4c/commit/c058e82c6a939f591059672dab08ddd0d4ce416e) which is roughly 2 years old.

Yet at the same time statistics of the issue indicates that some reproduction rate is still there.

This metadata inconsistency makes me wondering whether the testcase 5609076764573696 itself reproduces (and the info about commit is wrong and stale); or whether there are some new testcases which were added into the group but are not somehow accessible in the interface for some reason.

Can you please check what's going on?

Contributor guide

Open the contributing guide

Research direction

Start with OSS-Fuzz testcase 5609076764573696 and compare its reported last-reproduced commit, c058e82c6a939f591059672dab08ddd0d4ce416e, with the current reproduction statistics. Check whether newer timeout testcases are grouped under the issue and whether the group icon is available. Done means documenting which metadata is current and why the issue is being reopened.

Written by the indexing model from the issue text.

Assessment

Domain
testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.