deepseek-ai / deepseek-ai/DeepSpec

Eagle3 fallback attention mask crashes with KV cache

Open
#33 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
7.1k
Forks
667
PR merge metrics
No merged PRs in 30d

Description

## Problem

`prepare_4d_causal_attention_mask` expects the padding-mask key dimension to match `kv_len`. In the Eagle3 TTT loop with a KV cache, `kv_len` grows with cached chunks, but the incoming 2D `attention_mask` still covers only the current chunk.

For `seq_len = 4` at cached step `k = 1`:

```text
causal mask: [1, 1, 4, 8]
expanded mask: [B, 1, 1, 4]
```

Adding these tensors fails on the key dimension.

## Impact

The non-flex fallback attention path crashes whenever cached Eagle3 TTT steps require `kv_len > seq_len`.

## Fix

Fixed in PR #30 by extending the 2D keep-mask to `kv_len` before converting it to the additive 4D mask. The final fix repeats the fixed chunk mask pattern, preserving padded cached columns instead of marking all cached positions valid.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.