The past_key_value in MPLUG doesn't seem to work.
- Dominant language
- Python
- Stars
- 2k
- Forks
- 301
- PR merge metrics
- No merged PRs in 30d
Description
I change the [code](https://github.com/alibaba/AliceMind/blob/a6d5afe5c272960ce61dc73adb0877bc94b5d664/mPLUG/models/predictor.py#L197) like this, and it didn't work.
```
kv_cache = None
prev_tokens = set()
max_length = int(max_length)
for step in range(max_length):
dec_feat_seq = self.model(alive_seq,
encoder_hidden_states = src_features,
encoder_attention_mask = attention_mask,
past_key_values = None,
use_cache = False,
return_dict = True,
reduction = 'none')
kv_cache = dec_feat_seq.past_key_values
```
Contributor guide
No contributing guide indexed for this repository
Research direction
Start in mPLUG/models/predictor.py around line 197 and trace how the generation loop handles past_key_values and use_cache. Reproduce the reported failure with the provided loop, then verify that cached decoder state is used correctly and generation completes as expected.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100