Is Multi-head Latent Attention used in DeepSeek-VL2 tiny?

Hi and thank you for your great work,

In the paper you published in Table 1 it says that the tiny version of the model uses the original Multi-Head Attention and not MLA:


However, when I run it locally I see the following module in the architecture:


which is part of the vision model, which makes it even more confusing for me as I thought that MLA is applied in the language model.

Could you please clarify what this module is, as it is not part of the original siglip-so400m-patch14-384, which is supposed to be your vision backbone.