up matrix is not involved?

Hi,

very roughly, MLA compresses the input into a latent tensor via DOWN matrix, caches the latent tensor, and then converts the latent tensor back to 'normal' QKVs via UP matrix before SDPA.

looks that FlashMLA does not accept the UP matrix in its parameters, and so the inputs of FlashMLA are 'normal' (MHA/GQA/MQA) QKVs?

imho, FlashMLA is expected to accept MLA caches and UP matrices etc, doing possible matrix absorb together with SDPA.