michaelfeil / michaelfeil/infinity

Optimum 2.0.0: Bettertransformer deprecated

Open
#649 5 comments 2 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
2.9k
Forks
206
PR merge metrics
No merged PRs in 30d

Description

### System Info

When install with pip install infinity[all] it will install optimum 2.0.0. Resulting in the following error when using the infinity cli.

```
infinity_emb v2 --help
Traceback (most recent call last):
File "/media/data4/yh/uvenv/infinity077/bin/infinity_emb", line 4, in
from infinity_emb.cli import cli
File "/media/data4/yh/uvenv/infinity077/lib/python3.12/site-packages/infinity_emb/__init__.py", line 7, in
from infinity_emb.engine import AsyncEmbeddingEngine, AsyncEngineArray # noqa: E402
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/media/data4/yh/uvenv/infinity077/lib/python3.12/site-packages/infinity_emb/engine.py", line 11, in
from infinity_emb.inference import (
File "/media/data4/yh/uvenv/infinity077/lib/python3.12/site-packages/infinity_emb/inference/__init__.py", line 4, in
from infinity_emb.inference.batch_handler import BatchHandler
File "/media/data4/yh/uvenv/infinity077/lib/python3.12/site-packages/infinity_emb/inference/batch_handler.py", line 39, in
from infinity_emb.transformer.utils import get_lengths_with_tokenize
File "/media/data4/yh/uvenv/infinity077/lib/python3.12/site-packages/infinity_emb/transformer/utils.py", line 9, in
from infinity_emb.transformer.classifier.torch import SentenceClassifier
File "/media/data4/yh/uvenv/infinity077/lib/python3.12/site-packages/infinity_emb/transformer/classifier/torch.py", line 8, in
from infinity_emb.transformer.acceleration import (
File "/media/data4/yh/uvenv/infinity077/lib/python3.12/site-packages/infinity_emb/transformer/acceleration.py", line 11, in
from optimum.bettertransformer import ( # type: ignore[import-untyped]
ModuleNotFoundError: No module named 'optimum.bettertransformer'
```

Optimum installed:
```
pip freeze | grep optimum
optimum==2.0.0
```

Checking the optimum released notes, https://github.com/huggingface/optimum/releases/tag/v2.0.0 bettertransformer has been deprecated and removed.

### Information

- [ ] Docker + cli
- [x] pip + cli
- [ ] pip + usage of Python interface

### Tasks

- [ ] An officially supported CLI command
- [ ] My own modifications

### Reproduction

Installed with

> pip install infinity-emb[all]

Failed to run cli as mentioned above.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with infinity_emb/transformer/acceleration.py, where the traceback imports optimum.bettertransformer, and review the Optimum 2.0.0 release notes linked in the issue. Reproduce the failure with pip install infinity-emb[all] and infinity_emb v2 --help; done means the CLI starts without the missing-module error.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, cli
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.