[FEATURE] Support DeepInfra as a native OpenAI-compatible LLM provider
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 58.8k
- Forks
- 8.5k
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 109
Description
Feature Area
Integration with external tools
Is your feature request related to a an existing bug? Please link it here.
N/A
Describe the solution you'd like
Add DeepInfra as native openai compatible provider, same way as openrouter, deepseek, cerebras and dashscope are in OPENAI_COMPATIBLE_PROVIDERS. Right now LLM(model="deepinfra/...") goes to LiteLLM and without crewai[litellm] installed it fails with ImportError. It should just work with DEEPINFRA_API_KEY set:
llm = LLM(model="deepinfra/deepseek-ai/DeepSeek-V4-Flash-0731")
Base URL is https://api.deepinfra.com/v1/openai, the key is DEEPINFRA_API_KEY (same name LiteLLM already uses), DEEPINFRA_BASE_URL for override. Our model ids are org/model, so the full reference have three segments - the routing must keep the part after deepinfra/ as is.
Nothing is needed on the transport side. We tested tool calling, streaming, structured outputs and usage reporting with custom_openai=True against our API and all works. The change is one registry entry, few lines in llm.py, tests and the docs accordion in en/ar/ko/pt-BR - same shape as #5042 and the Eden AI PR #7049.
We (DeepInfra) maintain the API and will maintain the integration. PR is ready, I will open it linked to this issue.
Describe alternatives you've considered
custom_openai=True with base_url - works today, but users must know the URL and can't use deepinfra/ prefix in agents.yaml. LiteLLM - extra dependency, and crewAI is moving away from it.
Additional context
Follow-ups after this one, as separate PRs: DeepInfra embeddings provider (like the openrouter one in #7127) and DeepInfra in the CLI provider picker.
Willingness to Contribute
Yes, I'd be happy to submit a pull request
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the OPENAI_COMPATIBLE_PROVIDERS registry and the related provider logic in llm.py, then compare the shapes used by #5042 and #7049. Review the existing tests and the documentation accordions in en, ar, ko, and pt-BR. Done means the DeepInfra model reference works with DEEPINFRA_API_KEY, including its multi-segment model id, and the integration is covered in tests and docs.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai, backend-api-design, documentation
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 68/100