NVIDIA-NeMo / NVIDIA-NeMo/Curator
Add optional s3fs / gcfs as optional dependencies
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.8k
- Forks
- 328
- Avg merge
- 4d 5h
- Merged PRs (30d)
- 30
Description
Is your feature request related to a problem? Please describe.
If customers want to use NeMo Curator on Cloud we should add optional groups as s3 or gc that installs the correct version inline with our existing fsspec installation. If users install themselves using pip, it might be possible that Curator needs fsspec=2024.12 while pip as an additional install of s3fs installs 2025.12
Describe the solution you'd like
A clear and concise description of what you want to happen.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The issue names no files or tests. Start by locating the dependency configuration and the existing fsspec optional-dependency entry; verify the requested s3 and gc groups and their version constraints. Done means both optional groups install compatible cloud filesystem dependencies without changing the default installation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- build-system, cloud
- Issue type
- Feature
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 48/100