Git Checkout might not work if there are multiple copies of a repo.
- Dominant language
- Python
- Stars
- 1.2k
- Forks
- 302
- PR merge metrics
- No merged PRs in 30d
Description
The `benchmark.yaml` file has a checkout hash that can be used to checkout a specific commit before running the benchmarks. We have observed that the checkout seems to fail spuriously in some benchmarks, like in `libjpeg-turbo_libjpeg_turbo_fuzzer`.
By [looking at the source code](https://github.com/google/fuzzbench/blob/7c70037a73d2cc66627c6109d53d20b3594f85c9/docker/benchmark-builder/checkout_commit.py#L92-L96), we found out that the code seems to go into each directory and try to checkout the specific commit, stopping at the first folder that succeeded.
This means that, in the case of libjpeg-turbo that has at least 3 folders (one for each branch), only one of them will be in the right commit. Furthermore, it seems like the order of the directory iteration is not stable, so it might change between runs.
We haven't looked to see which other benchmarks are affected.
Contributor guide
Assessment
This issue has not been assessed yet.