google / google/oss-fuzz

Python Coverage Question

Open
#11,215 6 comments 0 reactions 0 assignees View on GitHub
Dominant language
Shell
Stars
12.6k
Forks
2.9k
Avg merge
2d 2h
Merged PRs (30d)
62

Description

Hello,

I'd appreciate some more clarity on how ClusterFuzz calculates Python coverage. If I run coverage locally, I get a coverage above 50% for a project I am working on. However, on ClusterFuzz, that same project has a coverage report at 37% due to the inclusion of extraneous site-packages used by the library being fuzzed.

So, my questions are the following
1. Is there a way to disclude certain libraries from the Python coverage report?
2. How is coverage "graded" for the >50% ideal integration standard? Is it based solely off library source files, library source files + dependencies, or the introspector report?

Thank you for your review and help!

Contributor guide

Open the contributing guide

Research direction

Start by reviewing how ClusterFuzz calculates Python coverage and how the >50% integration standard is evaluated. Compare the local report with the ClusterFuzz report, focusing on whether dependencies and introspector results are included. Done means documenting the coverage inputs and whether libraries can be excluded.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing-qa
Issue type
Documentation
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.