huggingface / huggingface/datatrove

Overloaded dependencies with `inference` feature

Open
#476 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
3.3k
Forks
302
Avg merge
2h 18m
Merged PRs (30d)
2

Description

## Summary

`InferenceRunner` currently appears to require installation of backend-specific inference servers through the `datatrove[inference]` extra, even when those backends are not used.

It would be cleaner to separate the base inference framework from the server-specific integrations.

## Problem

Today, the `inference` extra seems to bundle multiple inference server dependencies together. In practice, that means users may need to install packages for backends they do not intend to use.

For example, a user may want to:

* use the base `InferenceRunner` abstractions
* target only remote / endpoint-based inference
* use a custom backend
* use only one local server implementation

But they still end up pulling in unrelated inference-server dependencies.

This creates a few issues:

* heavier installs
* more dependency conflicts
* harder environment management
* unnecessary platform / CUDA coupling for users who are not using that backend

## Proposed change

Split the extras so that the core inference framework is separate from the backend implementations.

Example:

* `datatrove[inference]` → base inference functionality only
* `datatrove[inference,vllm]` → base inference + vLLM support
* `datatrove[inference,sglang]` → base inference + SGLang support

Potentially also:

* `datatrove[inference,all]` → install all supported inference backends

## Expected behavior

This would let users choose only the backend they need:

```bash
pip install "datatrove[inference]"
pip install "datatrove[inference,vllm]"
pip install "datatrove[inference,sglang]"
```

Or, if desired:

```bash
pip install "datatrove[inference,all]"
```

## Benefits

* cleaner separation of concerns
* lighter installs for endpoint-only or custom-backend users
* fewer dependency/version conflicts
* easier deployment in constrained environments
* clearer mapping between runtime backend choice and installed dependencies

## Notes

This seems especially relevant because `InferenceRunner` conceptually supports multiple server types, but installation is currently more tightly coupled to specific backend implementations than necessary.

Happy to help sketch a packaging layout if this direction sounds reasonable.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.