AnswerDotAI / AnswerDotAI/rerankers
Model loading is broken with transformers >= 5
- Dominant language
- Python
- Stars
- 1.6k
- Forks
- 104
- PR merge metrics
- No merged PRs in 30d
Description
### Issue
Running this code with the newer versions of transformers v5.x
```python
import rerankers
rerankers.Reranker("colbert")
```
throws the following error: `AttributeError: 'ColBERTModel' object has no attribute 'all_tied_weights_keys'`.
(See full stack trace bellow)
Setup for reproduction:
```bash
# mkdir test-issue; cd test-issue
uv init
uv add rerankers[transformers]
uv sync
# Add the above code in main.py
uv run main.py
```
This installs the newest version of transformers (currently 5.1.0).
See here: https://github.com/AnswerDotAI/rerankers/blob/5b9cbb073335b0246bc33e939600f51f934c460d/pyproject.toml#L67
The error occurs with both transformers v5.0 and v5.1
### Expected behavior
To load and use the model as usual.
### Cause
As far as I can tell, transformers v5.x has this `PreTrainedModel.post_init()` method that they now expect to be called in the `__init__()` method of derived classes. This method creates the `all_tied_weights_keys` attribute from the error.
See here: https://github.com/huggingface/transformers/blob/f73a4db3a0bcf6523e9bfdaaf4afe81dffba4da8/src/transformers/modeling_utils.py#L1298
I've seen other people submitting issues for this problem:
- https://github.com/huggingface/transformers/issues/43883
- https://github.com/huggingface/transformers/issues/42832
Here is the comment in which one of maintainers explains the problem and mentions their expectations: https://github.com/huggingface/transformers/issues/43883#issuecomment-3891672810
Currently, transformers version v4.57 does not raise this error. (I guess any (recent) version <5 works)
For reference, here is the `post_init()` method from transformers v4.57.6: https://github.com/huggingface/transformers/blob/753d61104116eefc8ffc977327b441ee0c8d599f/src/transformers/modeling_utils.py#L2103
### Stack trace
```
Loading default colbert model for language en
Default Model: colbert-ir/colbertv2.0
Loading ColBERTRanker model colbert-ir/colbertv2.0 (this message can be suppressed by setting verbose=0)
No device set
Using device cpu
No dtype set
Using dtype torch.float32
Loading model colbert-ir/colbertv2.0, this might take a while...
Linear Dim set to: 128 for downcasting
Loading weights: 100%|███████████████████████████████████████████████████████████████████████| 200/200 [00:00<00:00, 2974.21it/s, Materializing param=linear.weight]
ColBERTModel LOAD REPORT from: colbert-ir/colbertv2.0
Key | Status | |
-----------------------------+------------+--+-
bert.embeddings.position_ids | UNEXPECTED | |
Notes:
- UNEXPECTED :can be ignored when loading from different task/architecture; not ok if you expect identical arch.
Traceback (most recent call last):
File "/home/vlad/Work/test/main.py", line 10, in
main()
~~~~^^
File "/home/vlad/Work/test/main.py", line 6, in main
rerankers.Reranker("colbert")
~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
File "/home/vlad/Work/test/.venv/lib/python3.13/site-packages/rerankers/reranker.py", line 238, in Reranker
return AVAILABLE_RANKERS[model_type](model_name, verbose=verbose, **kwargs)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vlad/Work/test/.venv/lib/python3.13/site-packages/rerankers/models/colbert_ranker.py", line 238, in __init__
ColBERTModel.from_pretrained(
~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
model_name,
^^^^^^^^^^^
verbose=self.verbose,
^^^^^^^^^^^^^^^^^^^^^
**model_kwargs
^^^^^^^^^^^^^^
)
^
File "/home/vlad/Work/test/.venv/lib/python3.13/site-packages/transformers/modeling_utils.py", line 4064, in from_pretrained
loading_info = cls._finalize_model_loading(model, load_config, loading_info)
File "/home/vlad/Work/test/.venv/lib/python3.13/site-packages/transformers/modeling_utils.py", line 4205, in _finalize_model_loading
model.mark_tied_weights_as_initialized()
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
File "/home/vlad/Work/test/.venv/lib/python3.13/site-packages/transformers/modeling_utils.py", line 4524, in mark_tied_weights_as_initialized
for tied_param in self.all_tied_weights_keys.keys():
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vlad/Work/test/.venv/lib/python3.13/site-packages/torch/nn/modules/module.py", line 1965, in __getattr__
raise AttributeError(
f"'{type(self).__name__}' object has no attribute '{name}'"
)
AttributeError: 'ColBERTModel' object has no attribute 'all_tied_weights_keys'. Did you mean: '_tied_weights_keys'?
```
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.