apache / apache/tvm

[Bug] 'tvm.relax.op.nn' has no attribute 'attention_bias'

Open
#17,486 3 comments 0 reactions 0 assignees View on GitHub
needs-triage type: bug
Dominant language
Python
Stars
13.7k
Forks
4k
Avg merge
2d 19h
Merged PRs (30d)
111

Description

### Actual behavior

```
@I.ir_module
class Module:
@R.function
def main(q: R.Tensor((4, 16, 32, 8), dtype="float32"), k: R.Tensor((4, 8, 32, 8), dtype="float32"), v: R.Tensor((4, 8, 32, 16), dtype="float32"), bias: R.Tensor((4, 32, 16, 8), dtype="float32")) -> R.Tensor((4, 16, 32, 16), dtype="float32"):
gv: R.Tensor((4, 16, 32, 16), dtype="float32") = R.nn.attention_bias(q, k, v, bias, scale=T.float32(0.10000000000000001), causal_mask="TopLeft", window_size=None)
return gv

error: module 'tvm.relax.op.nn' has no attribute 'attention_bias'
--> :9:58
|
9 | gv: R.Tensor((4, 16, 32, 16), dtype="float32") = R.nn.attention_bias(q, k, v, bias, scale=T.float32(0.10000000000000001), causal_mask="TopLeft", window_size=None)
| ^^^^^^^^^^^^^^^^^^^
```

### Steps to reproduce
```
irs= """# from tvm.script import ir as I
# from tvm.script import tir as T
# from tvm.script import relax as R

@I.ir_module
class Module:
@R.function
def main(q: R.Tensor((4, 16, 32, 8), dtype="float32"), k: R.Tensor((4, 8, 32, 8), dtype="float32"), v: R.Tensor((4, 8, 32, 16), dtype="float32"), bias: R.Tensor((4, 32, 16, 8), dtype="float32")) -> R.Tensor((4, 16, 32, 16), dtype="float32"):
gv: R.Tensor((4, 16, 32, 16), dtype="float32") = R.nn.attention(q, k, v, bias, scale=T.float32(0.10000000000000001), causal_mask="TopLeft", window_size=None)
return gv

import tvm
mod = tvm.script.from_source(irs)
mod.show()
mod_new = tvm.script.from_source(mod.script()) # crash!
"""
```

cc @Lunderberg @junrushao @Hzfengsy @tqchen

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by running the provided tvm.script.from_source round-trip reproducer and compare the R.nn.attention call in the input with the generated R.nn.attention_bias call and reported tvm.relax.op.nn attribute error. Trace the script serialization and parsing path; done means the module round-trips without crashing and the attention operation remains valid.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
compilers
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.