crewAIInc / crewAIInc/crewAI

[FEATURE] Support llmman as an OpenAI-compatible provider

Open
#7,216 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

vendor-pitch
Dominant language
Python
Stars
58.8k
Forks
8.5k
Avg merge
1d 15h
Merged PRs (30d)
109

Description

Feature Area: Integration with external tools

Problem

llmman runs local models distributed as OCI artifacts and serves an OpenAI-compatible API on port 17434. CrewAI already ships first-class support for the other local OpenAI-compatible servers (Ollama, vLLM), but there is no way to point a crew at llmman without hand-configuring base_url on every LLM.

Proposal

Register llmman alongside the existing entries:

  • A ProviderConfig in OPENAI_COMPATIBLE_PROVIDERS (LLMMAN_HOST, no API key required).
  • Runtime routing in crewai.llm.LLM so LLM(model="llmman/qwen3.8") and LLM(model="qwen3.8", provider="llmman") reach OpenAICompatibleCompletion.
  • CLI scaffolding entries (PROVIDERS, ENV_VARS, MODELS) so crewai create offers it.

LLMMAN_HOST is a bare host:port value, so it needs the same /v1 base-URL normalization already applied to OLLAMA_HOST.

Additional context

Happy to implement this.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with OPENAI_COMPATIBLE_PROVIDERS and the existing OLLAMA_HOST normalization, then trace provider routing in crewai.llm.LLM. Review the CLI PROVIDERS, ENV_VARS, and MODELS entries. Done means both model/provider forms route to OpenAICompatibleCompletion and crewai create exposes llmman with LLMMAN_HOST.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ai, backend-api-design, cli
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.