Presubmit failure: AssertionErrors
- Dominant language
- Python
- Stars
- 1.2k
- Forks
- 302
- PR merge metrics
- No merged PRs in 30d
Description
Hi, any ideas on how to fix the following assertion errors? (I get them when I run `make presubmit`):
`=================================== FAILURES ===================================`
`____________________ test_benchmark_rank_by_stat_test_wins _____________________`
`...
[make_presubmit_log.txt](https://github.com/google/fuzzbench/files/6251987/make_presubmit_log.txt)
`
`> assert ranking.equals(expected_ranking)`
`E assert False`
`E + where False = (libfuzzer 0\nafl 0\ndtype: int64)`
`E + where = fuzzer\nafl 0\nlibfuzzer 0\nName: stat wins, dtype: int64.equals`
`analysis/test_data_utils.py:296: AssertionError`
`______________________ test_experiment_rank_by_num_firsts ______________________`
`def test_experiment_rank_by_num_firsts():
experiment_df = create_experiment_data()
snapshots_df = data_utils.get_experiment_snapshots(experiment_df)
ranking = data_utils.experiment_level_ranking(
snapshots_df, data_utils.benchmark_rank_by_median,
data_utils.experiment_rank_by_num_firsts)`
`expected_ranking = pd.Series(index=['libfuzzer', 'afl'], data=[1.0, 1.0])`
`> assert ranking.equals(expected_ranking)`
`E assert False`
`E + where False = (libfuzzer 1.0\nafl 1.0\ndtype: float64)`
`E + where = fuzzer\nafl 1.0\nlibfuzzer 1.0\nName: number of wins, dtype: float64.equals`
`analysis/test_data_utils.py:339: AssertionError`
`__________________________ test_fuzz_function_errors ___________________________`
`def test_fuzz_function_errors():
"""This test calls fuzzer_module.fuzz() under circumstances in
which it should throw an exception. If the call exceptions, the
test passes, otherwise the test fails. This ensures that we can
properly detect failures during fuzzing."""
for fuzzer_module in _get_all_fuzzer_modules():
with pytest.raises(Exception) as error, Patcher():
fuzzer_module.fuzz('/input-corpus', '/output-corpus',
'/target-binary')`
` # Type error probably means module is doing something else wrong,`
` # so fail if we see one. If that is not the case than this assert`
` # should be removed.`
`> assert not isinstance(error.value, TypeError)`
`E assert not True
E + where True = isinstance(TypeError("read() missing 1 required positional argument: 'numBytes'"), TypeError)
E + where TypeError("read() missing 1 required positional argument: 'numBytes'") = .value`
`fuzzers/test_fuzzers.py:81: AssertionError`
Contributor guide
Assessment
This issue has not been assessed yet.