apache / apache/texera

Add LiteLLM proxy helm chart support for AI agent integration

Open
#4,108 0 comments 0 reactions 1 assignee Claimed by @bobbai00 View on GitHub
Dominant language
Scala
Stars
314
Forks
187
Avg merge
1d 21h
Merged PRs (30d)
214

Description

### Feature Summary

Add helm chart support for deploying LiteLLM proxy as part of the Texera Kubernetes deployment. This enables AI agent functionality by providing a unified LLM API gateway that can route requests to various LLM providers (Anthropic, OpenAI, etc.).

The LiteLLM proxy acts as a central point for:
- Managing API keys for multiple LLM providers
- Providing a unified OpenAI-compatible API for all models
- Tracking usage and spend across different models
- Enabling the access control service to interact with LLM models for agent features

### Proposed Solution or Design

The implementation includes:

1. **New Helm Templates:**
- `litellm-config.yaml` - ConfigMap for LiteLLM configuration
- `litellm-deployment.yaml` - Deployment for the LiteLLM proxy container
- `litellm-secret.yaml` - Secret for API keys (Anthropic, OpenAI) and database credentials
- `litellm-service.yaml` - ClusterIP service for internal access
- `postgresql-litellm-persistence.yaml` - PV/PVC for LiteLLM's PostgreSQL database

2. **Chart Dependencies:**
- Add postgresql-litellm as an aliased postgresql dependency for LiteLLM's persistence

3. **Service Integration:**
- Update access-control-service deployment to include LITELLM_MASTER_KEY and LITELLM_BASE_URL environment variables
- Add ExternalName services for cross-namespace access
- Add ingress paths for `/api/models` and `/api/chat` endpoints

4. **Configuration:**
- Add `litellm` section in values.yaml with configurable image, resources, API keys, and persistence settings
- Add `postgresql-litellm` section for the dedicated LiteLLM database

### Impact / Priority

(P2)Medium – useful enhancement

### Affected Area

- Deployment / Infrastructure

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.