michaelfeil / michaelfeil/infinity
Support for nomic-ai/colnomic-embed-multimodal-7b
- Dominant language
- Python
- Stars
- 2.9k
- Forks
- 206
- PR merge metrics
- No merged PRs in 30d
Description
### Model description
Getting this config.json error
`INFO: Started server process [1]
INFO: Waiting for application startup.
INFO 2025-05-20 18:28:25,010 infinity_emb INFO: infinity_server.py:84
Creating 1engines:
engines=['nomic-ai/colnomic-embed-multimodal-7b']
INFO 2025-05-20 18:28:25,013 infinity_emb INFO: Anonymized telemetry.py:30
telemetry can be disabled via environment variable
`DO_NOT_TRACK=1`.
INFO 2025-05-20 18:28:25,019 infinity_emb INFO: select_model.py:64
model=`nomic-ai/colnomic-embed-multimodal-7b`
selected, using engine=`torch` and device=`None`
ERROR: Traceback (most recent call last):
File "/app/.venv/lib/python3.10/site-packages/huggingface_hub/utils/_http.py", line 406, in hf_raise_for_status
response.raise_for_status()
File "/app/.venv/lib/python3.10/site-packages/requests/models.py", line 1024, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://huggingface.co/nomic-ai/colnomic-embed-multimodal-7b/resolve/main/config.json
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/app/.venv/lib/python3.10/site-packages/starlette/routing.py", line 693, in lifespan
async with self.lifespan_context(app) as maybe_state:
File "/usr/lib/python3.10/contextlib.py", line 199, in __aenter__
return await anext(self.gen)
File "/app/infinity_emb/infinity_server.py", line 88, in lifespan
app.engine_array = AsyncEngineArray.from_args(engine_args_list) # type: ignore
File "/app/infinity_emb/engine.py", line 306, in from_args
return cls(engines=tuple(engines))
File "/app/infinity_emb/engine.py", line 71, in from_args
engine = cls(**engine_args.to_dict(), _show_deprecation_warning=False)
File "/app/infinity_emb/engine.py", line 56, in __init__
self._model_replicas, self._min_inference_t, self._max_inference_t = select_model(
File "/app/infinity_emb/inference/select_model.py", line 71, in select_model
unloaded_engine = get_engine_type_from_config(engine_args)
File "/app/infinity_emb/inference/select_model.py", line 36, in get_engine_type_from_config
config_path = hf_hub_download(
File "/app/.venv/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn
return fn(*args, **kwargs)
File "/app/.venv/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 862, in hf_hub_download
return _hf_hub_download_to_cache_dir(
File "/app/.venv/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 925, in _hf_hub_download_to_cache_dir
(url_to_download, etag, commit_hash, expected_size, head_call_error) = _get_metadata_or_catch_error(
File "/app/.venv/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 1376, in _get_metadata_or_catch_error
metadata = get_hf_file_metadata(
File "/app/.venv/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn
return fn(*args, **kwargs)
File "/app/.venv/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 1296, in get_hf_file_metadata
r = _request_wrapper(
File "/app/.venv/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 277, in _request_wrapper
response = _request_wrapper(
File "/app/.venv/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 301, in _request_wrapper
hf_raise_for_status(response)
File "/app/.venv/lib/python3.10/site-packages/huggingface_hub/utils/_http.py", line 417, in hf_raise_for_status
raise _format(EntryNotFoundError, message, response) from e
huggingface_hub.errors.EntryNotFoundError: 404 Client Error. (Request ID: Root=1-682cc9c9-3ab935887bf5e52b5a91b47b;7b8027d0-5750-4516-af73-32cea0296e33)
Entry Not Found for url: https://huggingface.co/nomic-ai/colnomic-embed-multimodal-7b/resolve/main/config.json.
ERROR: Application startup failed. Exiting.`
### Open source status & huggingface transformers.
- [x] The model implementation is available on transformers
- [x] The model weights are available on huggingface-hub
- [x] I verified that the model is currently not running in the latest version `pip install infinity_emb[all] --upgrade`
- [x] I made the authors of the model aware that I want to use it with infinity_emb & check if they are aware of the issue.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start in infinity_emb/inference/select_model.py, especially get_engine_type_from_config, and trace how infinity_emb/engine.py selects an engine from a Hugging Face config. Check the model's config and Transformers support before determining the integration scope. Done means the named model can pass startup model selection and run successfully through the supported serving path.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- huggingface, python
- Domain
- backend, machine-learning
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100