bedapub / bedapub/besca

LFS quota exceeded

Open
#323 27 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
50
Forks
17
Avg merge
1d 4h
Merged PRs (30d)
3

Description

Hi all,

Besca contains references to some LFS files. As far as I understand, when someone git clones the repo, LFS files are also pulled.

We currently are over bandwidth, which means that none of the bedapub projects can push LFS files. The error I was getting:
```
LFS files into bedapub repo, but I was getting this message:
batch response: This repository is over its data quota. Account responsible for LFS bandwidth should purchase more data packs to restore access.
error: failed to push some refs
```

As far as I understand, these files are only in history and are now stored elsewhere.

What I propose currently is to rewrite history and remove all the references to those files. The upside of this is that in one month LFS hopefully will be usable again. They will be still in the storage, but at least bedapub will have 700 Mb for other projects to use.

When history is rewrited, one should make sure to clone the repository again, otherwise there's a risk to accidentally push large files again.

An alternative solution would be to get budget for LFS.

What do you think?

CC @hatjek @kohleman @swalpe

Here's the list of references:
```
$ git lfs ls-files --deleted --all
75f416ec45 - besca/datasets/data/pbmc_storage_processed_downsampled.h5ad
81fae1e6df - besca/datasets/data/pbmc_storage_raw_downsampled.h5ad
e325371e52 - besca/datasets/data/pbmc3k_filtered.h5ad
ba1c7d7ade - besca/datasets/data/pbmc3k_processed.h5ad
bd409cc9c1 - besca/datasets/data/pbmc3k_raw.h5ad
49381ba23b - besca/datasets/data/pbmc_storage_raw_downsampled.h5ad
```

quota usage
image

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the listed deleted LFS references under besca/datasets/data and reproduce the issue with git lfs ls-files --deleted --all. Review the 27-comment discussion to determine whether history rewriting or additional LFS budget was chosen. Done means the repository no longer consumes bandwidth from those historical files and the agreed repository state is documented for a fresh clone.

Written by the indexing model from the issue text.

Assessment

Tech stack
git, github
Domain
devops, infrastructure
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.