.gc within zarr_sync is not "in effect"?
- Dominant language
- Python
- Stars
- 15
- Forks
- 2
- PR merge metrics
- No merged PRs in 30d
Description
while assessing what consumes space I spotted that some zarrs have huge number of loose objects, which seems to be packed or cleaned up with `git gc` call. But it seems that we do call `git gc` after syncing zarr: https://github.com/dandi/dandisets/blob/306e1ae3732e8f44fbf79ad4b44e9bb7bf07dc2b/tools/backups2datalad/zarr.py#L577
so it is not clear why we still have the case (may be failed syncs which never got `git gc` ran, only me doing `git reset --hard`?). I am still collecting stats across all zarrs under `/tmp/dandizarrs-inodes.out` but here is an example of one and effect of running `git gc` now:
```shell
dandi@drogon:/mnt/backup/dandi/dandizarrs/0cf3c734-4bb2-47d1-8e6a-528a3739af01$ du -scm --inodes .git/objects/; git gc; du -scm --inodes .git/objects/
232704 .git/objects/
232704 total
Enumerating objects: 541506, done.
Counting objects: 100% (541506/541506), done.
Delta compression using up to 4 threads
Compressing objects: 100% (469070/469070), done.
Writing objects: 100% (541506/541506), done.
Total 541506 (delta 224707), reused 156961 (delta 72419), pack-reused 0
Removing duplicate objects: 100% (256/256), done.
7 .git/objects/
7 total
```
so we went from 230k to 7 inodes in .git/objects.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.