nnx.make_causal_mask() usage
Open
- Dominant language
- Jupyter Notebook
- Stars
- 7.3k
- Forks
- 833
- Avg merge
- 5h 11m
- Merged PRs (30d)
- 5
Description
So this is a follow-up on #4290 (@cgarciae). For building a causal LM, I need to use causal masking. Here is my attempt (by adding a single line using the code from #4290:
```
batch_size = 2
seqlen = 40
emb_size = 256
x = jnp.ones((batch_size, seqlen, emb_size))
mha = nnx.MultiHeadAttention(
in_features=emb_size, num_heads=2, decode=True, rngs=nnx.Rngs(0)
)
shape = x.shape
for i in range(seqlen): # iterate all tokens
y = mha(inputs_q=x[:, i : i + 1],
mask=nnx.make_causal_mask(x[:, i : i + 1])) #newly added
```
The error I got is:
```
AssertionError: masks must have same rank: (5, 4)
```
I cannot make sense of this error :(
Contributor guide
Assessment
This issue has not been assessed yet.