crewAIInc / crewAIInc/crewAI

[FEATURE] Support DeepInfra as a native OpenAI-compatible LLM provider

Open
#7,455 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

feature-request vendor-pitch
Dominant language
Python
Stars
58.8k
Forks
8.5k
Avg merge
1d 15h
Merged PRs (30d)
109

Description

Feature Area

Integration with external tools

Is your feature request related to a an existing bug? Please link it here.

N/A

Describe the solution you'd like

Add DeepInfra as native openai compatible provider, same way as openrouter, deepseek, cerebras and dashscope are in OPENAI_COMPATIBLE_PROVIDERS. Right now LLM(model="deepinfra/...") goes to LiteLLM and without crewai[litellm] installed it fails with ImportError. It should just work with DEEPINFRA_API_KEY set:

llm = LLM(model="deepinfra/deepseek-ai/DeepSeek-V4-Flash-0731")

Base URL is https://api.deepinfra.com/v1/openai, the key is DEEPINFRA_API_KEY (same name LiteLLM already uses), DEEPINFRA_BASE_URL for override. Our model ids are org/model, so the full reference have three segments - the routing must keep the part after deepinfra/ as is.

Nothing is needed on the transport side. We tested tool calling, streaming, structured outputs and usage reporting with custom_openai=True against our API and all works. The change is one registry entry, few lines in llm.py, tests and the docs accordion in en/ar/ko/pt-BR - same shape as #5042 and the Eden AI PR #7049.

We (DeepInfra) maintain the API and will maintain the integration. PR is ready, I will open it linked to this issue.

Describe alternatives you've considered

custom_openai=True with base_url - works today, but users must know the URL and can't use deepinfra/ prefix in agents.yaml. LiteLLM - extra dependency, and crewAI is moving away from it.

Additional context

Follow-ups after this one, as separate PRs: DeepInfra embeddings provider (like the openrouter one in #7127) and DeepInfra in the CLI provider picker.

Willingness to Contribute

Yes, I'd be happy to submit a pull request

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the OPENAI_COMPATIBLE_PROVIDERS registry and the related provider logic in llm.py, then compare the shapes used by #5042 and #7049. Review the existing tests and the documentation accordions in en, ar, ko, and pt-BR. Done means the DeepInfra model reference works with DEEPINFRA_API_KEY, including its multi-segment model id, and the integration is covered in tests and docs.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ai, backend-api-design, documentation
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.