alibaba / alibaba/AliceMind

The past_key_value in MPLUG doesn't seem to work.

Open
#94 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
2k
Forks
301
PR merge metrics
No merged PRs in 30d

Description

I change the [code](https://github.com/alibaba/AliceMind/blob/a6d5afe5c272960ce61dc73adb0877bc94b5d664/mPLUG/models/predictor.py#L197) like this, and it didn't work.
```
kv_cache = None
prev_tokens = set()
max_length = int(max_length)
for step in range(max_length):
dec_feat_seq = self.model(alive_seq,
encoder_hidden_states = src_features,
encoder_attention_mask = attention_mask,
past_key_values = None,
use_cache = False,
return_dict = True,
reduction = 'none')
kv_cache = dec_feat_seq.past_key_values
```

Contributor guide

No contributing guide indexed for this repository

Research direction

Start in mPLUG/models/predictor.py around line 197 and trace how the generation loop handles past_key_values and use_cache. Reproduce the reported failure with the provided loop, then verify that cached decoder state is used correctly and generation completes as expected.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.