fsspec / fsspec/filesystem_spec
Reading Files from Git LFS Repo
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.4k
- Forks
- 490
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 38
Description
Hey,
I'm trying to read files from a GitHub LFS Repo (https://github.com/openai/dalle3-eval-samples/tree/main) but only get the pointers to the actual large files (the images in the repo), instead of the binaries.
Is there any way of reading these files from an LFS repo with fsspec?
My current testing code is:
import fsspec
github_repo = fsspec.get_mapper("github://openai:dalle3-eval-samples@main")
for file_name in github_repo:
file = github_repo[file_name]
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the provided fsspec.get_mapper("github://openai:dalle3-eval-samples@main") reproducer and inspect how the GitHub filesystem handles files from the repository. Reproduce the pointer-versus-binary behavior against the linked Git LFS repository; done means reading an LFS-backed image returns its binary contents rather than the pointer text, with coverage for that behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- git, github, python
- Domain
- backend
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100