deepseek-ai / deepseek-ai/DeepSeek-OCR

Issue: Incorrect Requirements - transformers 4.51.2 Incompatible with LlamaFlashAttention2 Import

Open
#182 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
23.9k
Forks
2.2k
PR merge metrics
No merged PRs in 30d

Description

# Issue: Incorrect Requirements - transformers Version Incompatible with LlamaFlashAttention2 Import

## Description

There is a critical version incompatibility in the official requirements specification. The model code attempts to import `LlamaFlashAttention2`, but this component is not available in the specified `transformers==4.46.3` (or even the newer `transformers==4.51.2`).

## Environment

- **Model**: deepseek-ai/DeepSeek-OCR
- **Transformers Version Tested**: 4.51.2
- **Official Requirements**:
- `torch==2.6.0`
- `transformers==4.46.3`
- `tokenizers==0.20.3`
- `flash-attn==2.7.3`

## Steps to Reproduce

1. Install the official requirements as specified
2. Attempt to load the model using standard initialization:
```python
tokenizer = AutoTokenizer.from_pretrained(model_dir, trust_remote_code=True)
model = AutoModel.from_pretrained(model_dir, trust_remote_code=True, use_safetensors=True)

The following import error occurs:
```
ImportError: cannot import name 'LlamaFlashAttention2' from 'transformers.models.llama.modeling_llama'
```

## Root Cause

The file modeling_deepseekv2.py contains:
``` python
from transformers.models.llama.modeling_llama import (
LlamaAttention,
LlamaFlashAttention2 # This import fails in transformers 4.46.3/4.51.2
)
```
The LlamaFlashAttention2 component is not available in the specified transformers versions, indicating a mismatch between the documented requirements and the actual code dependencies.

## Temporary Workaround
1. Modify the import in modeling_deepseekv2.py:

``` python
from transformers.models.llama.modeling_llama import (
LlamaAttention,
# LlamaFlashAttention2 # Commented out due to version incompatibility
)
```

2. Use eager attention implementation when loading the model:
``` python
tokenizer = AutoTokenizer.from_pretrained(model_dir, trust_remote_code=True)
model = AutoModel.from_pretrained(
model_dir,
attn_implementation="eager", # Force eager attention as workaround
trust_remote_code=True,
use_safetensors=True
)
```
## Questions/Suggestions
What is the correct transformers version that actually supports LlamaFlashAttention2?

Please update the official requirements to specify compatible versions.

Consider adding version checks or fallback mechanisms in the model code to handle such incompatibilities gracefully.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.