support for siglip 2 models

Model description

Hi,
many thanks for your great work bringing infinity-emb into life which solved a ton of problems I had plus a lot of time!
However, I tried to switch ftom siglip to siglip 2 models since they outperform the old branch, but inifinity-emb crashes and complains about a missing architecture field in config.json apparently:

jo@aibox:~/rf$ docker compose logs infinity-emb
infinity-emb-1 | INFO: Started server process [1]
infinity-emb-1 | INFO: Waiting for application startup.
infinity-emb-1 | INFO 2025-05-05 15:11:06,732 infinity_emb INFO: infinity_server.py:84
infinity-emb-1 | Creating 1engines:
infinity-emb-1 | engines=['google/siglip2-large-patch16-384']
infinity-emb-1 | INFO 2025-05-05 15:11:06,735 infinity_emb INFO: Anonymized telemetry.py:30
infinity-emb-1 | telemetry can be disabled via environment variable
infinity-emb-1 | DO_NOT_TRACK=1.
infinity-emb-1 | INFO 2025-05-05 15:11:06,740 infinity_emb INFO: select_model.py:64
infinity-emb-1 | model=google/siglip2-large-patch16-384 selected,
infinity-emb-1 | using engine=torch and device=cuda
infinity-emb-1 | ERROR: Traceback (most recent call last):
infinity-emb-1 | File "/app/.venv/lib/python3.10/site-packages/starlette/routing.py", line 693, in lifespan
infinity-emb-1 | async with self.lifespan_context(app) as maybe_state:
infinity-emb-1 | File "/usr/lib/python3.10/contextlib.py", line 199, in aenter
infinity-emb-1 | return await anext(self.gen)
infinity-emb-1 | File "/app/infinity_emb/infinity_server.py", line 88, in lifespan
infinity-emb-1 | app.engine_array = AsyncEngineArray.from_args(engine_args_list) # type: ignore
infinity-emb-1 | File "/app/infinity_emb/engine.py", line 306, in from_args
infinity-emb-1 | return cls(engines=tuple(engines))
infinity-emb-1 | File "/app/infinity_emb/engine.py", line 71, in from_args
infinity-emb-1 | engine = cls(**engine_args.to_dict(), _show_deprecation_warning=False)
infinity-emb-1 | File "/app/infinity_emb/engine.py", line 56, in init
infinity-emb-1 | self._model_replicas, self._min_inference_t, self._max_inference_t = select_model(
infinity-emb-1 | File "/app/infinity_emb/inference/select_model.py", line 81, in select_model
infinity-emb-1 | loaded_engine = unloaded_engine.value(engine_args=engine_args_copy)
infinity-emb-1 | File "/app/infinity_emb/transformer/vision/torch_vision.py", line 47, in init
infinity-emb-1 | self.is_colipali = config.architectures[0] in IMAGE_COL_MODELS
infinity-emb-1 | TypeError: 'NoneType' object is not subscriptable
infinity-emb-1 |
infinity-emb-1 | ERROR: Application startup failed. Exiting.

Any chance to add model compatibilty here?

something else I noted, not related to this, is that if I use --url-prefix /v1 in my v2 command, it also adds the prefix to rerank. Openai compatibile endpoints are /rerank and /v1/embeddings, so I need to run 2 instances of infinity-emb - which works fine and this is just meant as a note, maybe you want to 'fix' this in a future release

Kind regards,
Josh

Open source status & huggingface transformers.

• The model implementation is available on transformers
• The model weights are available on huggingface-hub
• I verified that the model is currently not running in the latest version pip install infinity_emb[all] --upgrade
• I made the authors of the model aware that I want to use it with infinity_emb & check if they are aware of the issue.

Start with infinity_emb/transformer/vision/torch_vision.py, where the traceback shows SigLIP 2 startup failing while reading config.architectures. Reproduce with google/siglip2-large-patch16-384 using the reported infinity-emb setup, then verify that the model starts and serves embeddings without regressing existing vision models.