huggingface / huggingface/huggingface-inference-toolkit

Inference Toolkit incompatible with transformers 5.x (is_tf_available import path changed) - due to strict transformers pin.

Open
#122 0 comments 2 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
97
Forks
28
Avg merge
6d 23m
Merged PRs (30d)
4

Description

Inference Endpoints fail when any dependency requires `transformers>=5.0` due to deprecated import paths.
```
ImportError: cannot import name 'is_tf_available' from 'transformers.file_utils'
```

The toolkit imports from `transformers.file_utils`, which was removed in transformers 5.0. These now live in `transformers.utils`.

**Example**:
Deploying [GLiNER](https://github.com/urchade/GLiNER) (`gliner>=0.2.23` requires `transformers>=4.57.3`) fails on Inference Endpoints even with a custom handler that doesn't use the toolkit directly.

1. Can the import paths be updated for transformers 5.x compatibility?
2. Why does the toolkit pin `transformers==4.51.3` exactly? This strict pinning will likely cause conflicts with many libraries.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.