Scorecard causing duplicate clones and massive spikes in disk usage for large repos
- Dominant language
- Python
- Stars
- 13
- Forks
- 17
- Avg merge
- 6h 59m
- Merged PRs (30d)
- 1
Description
> [!NOTE]
> Migrated from [augurlabs/augur#3699](https://github.com/augurlabs/augur/issues/3699)
> Originally opened by `@MoralCode` on 2026-02-15
---
From the grafana monitoring we see massive, temporary spikes in disk usage, sometimes into the tens of GB in size.
When run within our current docker configuration, these writes are saved to a new image layer in the main augur container. This layer gets so big that it fills up the root filesystem partition (where docker stores its data by default), causing augur and everything else that needs space in this partition, to fail.
Tracing the source
Using tools like `du` and `df` (linux commands) i found that one /var/lib/docker/overlay2 layer/directory was taking up ~all our root disk space (and was consuming like 60 GB)
using a [script i found online](https://www.onedotlab.com/blog/2026/01/find-docker-overlay-owner), i was able to map this layer id to the container that was creating it (augurs main container)
Shelling into that container and repeating the first set of tools i saw that most of this disk space was in /tmp within the augur container.
in /tmp there were ~5 folders named repo############### (random numbers too high to be repo IDs) containing either a massive githubrepo##########.tar.gz or what looks to be the actual contents of a repo (https://github.com/home-assistant/core was the one i saw, although it was missing a .git folder so maybe we were looking at a fork.) this repo has 100k commits.
The naming of the files that were filling up the disk line up with this line in the scorecard source code:
https://github.com/ossf/scorecard/blob/4eda2f56f70fa619b8785a63deb68aac8e3538ff/clients/githubrepo/tarball.go#L39
When we call scorecard, we are only passing it the repo url.
https://github.com/chaoss/augur/blob/892c635694e9c48f5eb964c494b19035e73db1cc/augur/tasks/git/dependency_tasks/core.py#L79
I suspect this is leading scorecard to try and clone the repo for itself.
This issue is not unique to docker, but docker exacerbates the issue by essentially storing data intended for /tmp in the root filesystem, which, on our instance, is not very big as everything (facade clones, postgres, etc) is stored in other filesystems/volumes attached to large disks.
Contributor guide
Assessment
This issue has not been assessed yet.