git2 holds open deleted packfiles, effectively leaking disk space
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 2.1k
- Forks
- 450
- Avg merge
- 11m
- Merged PRs (30d)
- 1
Description
We have a long-running daemon that does some stuff with a repository.
We noticed that the disk backing the repository was "full":
thing@thing-0:/var/cache/thing$ du -chs .
7.3G .
7.3G total
thing@thing-0:/var/cache/thing$ df -h .
Filesystem Size Used Avail Use% Mounted on
/dev/sdb 16G 16G 0 100% /var/cache/thing
That might see illogical, but that's a hint that deleted files are being held open. If we ls our daemon's open files, that confirms it:
0 -> /dev/null
1 -> 'pipe:[16731674]'
10 -> 'socket:[16737720]'
11 -> 'anon_inode:[eventpoll]'
12 -> 'anon_inode:[eventfd]'
13 -> 'anon_inode:[eventpoll]'
14 -> 'anon_inode:[eventfd]'
15 -> 'anon_inode:[eventpoll]'
16 -> 'socket:[16729776]'
17 -> 'anon_inode:[eventpoll]'
18 -> 'anon_inode:[eventfd]'
19 -> 'anon_inode:[eventpoll]'
2 -> 'pipe:[16731675]'
20 -> 'socket:[16729776]'
21 -> '/var/cache/thing/monorepo/.git/objects/pack/pack-a02eda73e77509f55e734b4f80f0e49c60093184.pack (deleted)'
22 -> '/var/cache/thing/environments/.git/objects/pack/pack-be204a28036cbc4cd01c2248a6ee2168a1989a96.pack (deleted)'
23 -> '/var/cache/thing/environments/.git/objects/pack/pack-6e09f3d300e72f084b05ba7bc1ccce40c10f582c.pack (deleted)'
24 -> '/var/cache/thing/monorepo/.git/objects/pack/pack-daec94cd70f17f2b1d3a638ce87cb4c086ce4629.pack (deleted)'
25 -> '/var/cache/thing/environments/.git/objects/pack/pack-2ad6f4562cd68fbf4ecc4d5af17579aa590e7263.pack (deleted)'
26 -> '/var/cache/thing/environments/.git/objects/pack/pack-aed3c72e24e8ec9a5db6b11bc2a81f071b5596f6.pack (deleted)'
27 -> '/var/cache/thing/environments/.git/objects/pack/pack-d4e7ed286a0e2e79cd36688ac33c5f98948e6b6a.pack (deleted)'
28 -> '/var/cache/thing/environments/.git/objects/pack/pack-6ffb344d987435d1cca7934ed89e7ddf9d6d79a2.pack (deleted)'
29 -> '/var/cache/thing/environments/.git/objects/pack/pack-09e7c333cc2b2dbb5b30550298b31f30ea4439c7.pack (deleted)'
3 -> 'anon_inode:[eventpoll]'
So, we can see here that we're holding a bunch of packfiles, which explains why the used space is so much higher than the du total.
We keep the git2::Repository object around: I think I didn't know if it was expensive to create, and we only have two repos, so we just create the git2::Repository objects are startup, and then keep them as global state.
But that seems to cause a "disk" leak, over repeated fetches & repacks.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the long-running daemon scenario with persistent git2::Repository objects and repeated fetches or repacks. Inspect the repository's open files for deleted packfiles and trace repository lifetime and packfile handling; done means repeated operations no longer retain deleted packfiles or exhaust disk space.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- git, rust
- Domain
- tooling
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100