huggingface / huggingface/datatrove
Overloaded dependencies with `inference` feature
- Dominant language
- Python
- Stars
- 3.3k
- Forks
- 302
- Avg merge
- 2h 18m
- Merged PRs (30d)
- 2
Description
## Summary
`InferenceRunner` currently appears to require installation of backend-specific inference servers through the `datatrove[inference]` extra, even when those backends are not used.
It would be cleaner to separate the base inference framework from the server-specific integrations.
## Problem
Today, the `inference` extra seems to bundle multiple inference server dependencies together. In practice, that means users may need to install packages for backends they do not intend to use.
For example, a user may want to:
* use the base `InferenceRunner` abstractions
* target only remote / endpoint-based inference
* use a custom backend
* use only one local server implementation
But they still end up pulling in unrelated inference-server dependencies.
This creates a few issues:
* heavier installs
* more dependency conflicts
* harder environment management
* unnecessary platform / CUDA coupling for users who are not using that backend
## Proposed change
Split the extras so that the core inference framework is separate from the backend implementations.
Example:
* `datatrove[inference]` → base inference functionality only
* `datatrove[inference,vllm]` → base inference + vLLM support
* `datatrove[inference,sglang]` → base inference + SGLang support
Potentially also:
* `datatrove[inference,all]` → install all supported inference backends
## Expected behavior
This would let users choose only the backend they need:
```bash
pip install "datatrove[inference]"
pip install "datatrove[inference,vllm]"
pip install "datatrove[inference,sglang]"
```
Or, if desired:
```bash
pip install "datatrove[inference,all]"
```
## Benefits
* cleaner separation of concerns
* lighter installs for endpoint-only or custom-backend users
* fewer dependency/version conflicts
* easier deployment in constrained environments
* clearer mapping between runtime backend choice and installed dependencies
## Notes
This seems especially relevant because `InferenceRunner` conceptually supports multiple server types, but installation is currently more tightly coupled to specific backend implementations than necessary.
Happy to help sketch a packaging layout if this direction sounds reasonable.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.