AI-Hypercomputer / AI-Hypercomputer/maxtext
`attend_dtype` not used
Ouverte
bug
- Langage dominant
- Python
- Étoiles
- 2.4k
- Forks
- 607
- Merge moyen
- 2 j 19 h
- PR mergées (30 j)
- 158
Description
[Here](https://github.com/google/maxtext/blob/5353a957594bd6cf316747cd5a327c163caca74f/MaxText/layers/embeddings.py#L102-L103) it seems that the hard-coded `bfloat16` is used instead of `attend_dtype`. Also `query` is not cast. I guess the correct behavior should be casting both `query` and `self.embedding` to `attend_dtype`?
Guide de contribution
Ouvrir le guide de contribution
Évaluation
Cette issue n'a pas encore été évaluée.