AI-Hypercomputer / AI-Hypercomputer/maxtext
`attend_dtype` not used
未关闭
bug
- 主要语言
- Python
- 星标
- 2.4k
- 派生
- 607
- 平均合并
- 2 天 19 小时
- 30 天内合并 PR
- 158
描述
[Here](https://github.com/google/maxtext/blob/5353a957594bd6cf316747cd5a327c163caca74f/MaxText/layers/embeddings.py#L102-L103) it seems that the hard-coded `bfloat16` is used instead of `attend_dtype`. Also `query` is not cast. I guess the correct behavior should be casting both `query` and `self.embedding` to `attend_dtype`?
贡献指南
评估
这个 Issue 还没有评估数据。