Add LiteLLM proxy helm chart support for AI agent integration
- Dominant language
- Scala
- Stars
- 314
- Forks
- 187
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 214
Description
### Feature Summary
Add helm chart support for deploying LiteLLM proxy as part of the Texera Kubernetes deployment. This enables AI agent functionality by providing a unified LLM API gateway that can route requests to various LLM providers (Anthropic, OpenAI, etc.).
The LiteLLM proxy acts as a central point for:
- Managing API keys for multiple LLM providers
- Providing a unified OpenAI-compatible API for all models
- Tracking usage and spend across different models
- Enabling the access control service to interact with LLM models for agent features
### Proposed Solution or Design
The implementation includes:
1. **New Helm Templates:**
- `litellm-config.yaml` - ConfigMap for LiteLLM configuration
- `litellm-deployment.yaml` - Deployment for the LiteLLM proxy container
- `litellm-secret.yaml` - Secret for API keys (Anthropic, OpenAI) and database credentials
- `litellm-service.yaml` - ClusterIP service for internal access
- `postgresql-litellm-persistence.yaml` - PV/PVC for LiteLLM's PostgreSQL database
2. **Chart Dependencies:**
- Add postgresql-litellm as an aliased postgresql dependency for LiteLLM's persistence
3. **Service Integration:**
- Update access-control-service deployment to include LITELLM_MASTER_KEY and LITELLM_BASE_URL environment variables
- Add ExternalName services for cross-namespace access
- Add ingress paths for `/api/models` and `/api/chat` endpoints
4. **Configuration:**
- Add `litellm` section in values.yaml with configurable image, resources, API keys, and persistence settings
- Add `postgresql-litellm` section for the dedicated LiteLLM database
### Impact / Priority
(P2)Medium – useful enhancement
### Affected Area
- Deployment / Infrastructure
Contributor guide
Assessment
This issue has not been assessed yet.