[Feature]: Multioutput models

What feature would you like to request?

In some cases our default model interface returning an array of numpy arrays is not enough
There are scenarios when we want to return more than that either in order to save resources or just because there is no other way.

An example of the first case is when models can return several types of embeddings using the same backbone but different head layers, e.g. baai/bge-m3.
Or when it is desired to obtain not only pooled embedding, but also token embeddings of the penultimate layer.

Another example is colpali-like models, it is not always possible to build index on colpali embeddings since of the huge resources consumption and thus it makes it difficult to use colpali as the first stage retriever.
Instead, first stage retrieval can be implemented via colpali embeddings pooled by either rows or columns.

In the particular case of colpali, it is possible to do the pooling after inference is completed and embeddings were returned, since the number of patches into which colpali preprocessor breaks an image is known beforehand.
However, with colpali-inspired models like colqwen, it is not possible to do the pooling afterwards, because the number of patches into which these models break an image is dynamic, thus, we need to know an image size to get the number of patches.

In order to handle these cases, we want to introduce new family of Multioutput classes with an interface allowing to return either dict or special model output objects.

Is there any additional information you would like to provide?

No response

The issue names no files or tests. Start by locating the default model interface and the inference path that returns arrays, then compare the embedding and pooling cases described; done should be a settled Multioutput interface supporting dict or model-output returns for the intended cases.