vllm-project / vllm-project/agentic-api

Deploy agentic-api as a remotely persistent, authenticated service

Open
#102 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
284
Forks
74
Avg merge
1d 17h
Merged PRs (30d)
93

Description

Summary

Make agentic-api easy to deploy as a production container on Kubernetes or a platform-as-a-service.

The deployment should support durable remote conversations and responses across restarts and replicas, with PostgreSQL as the production database and an external OpenAI-compatible/vLLM inference backend.

Proposed architecture

Client
  -> OIDC-authenticated application boundary
  -> agentic-api container
       -> managed PostgreSQL
       -> OpenAI-compatible inference service

The gateway remains a single Rust HTTP/WebSocket service. The container should run the gateway only; integrated serve mode can remain useful for local development, while hosted deployments point LLM_API_BASE at a separately managed inference service.

Authentication and authorization

Prioritize compatibility with Codex and Claude Code before designing application-issued API-key management.

  • Preserve standard Authorization: Bearer ... handling.
  • Preserve Anthropic-compatible x-api-key handling where required.
  • Pass client credentials to the configured upstream without inventing a new token format.
  • Defer API-key generation, storage, and management until the client integration path is clearer.
  • Treat OIDC as a possible future browser/admin or external-auth-proxy concern, not a prerequisite for CLI compatibility.
  • Enforce the authenticated identity/tenant on every conversation, response, and item read/write once the identity boundary is defined.

Work breakdown

Current repository anchors

  • DATABASE_URL and PostgreSQL support already exist in the Rust server/core.
  • /health and /ready endpoints already exist.
  • Stateful storage already uses conversations, responses, items, and tenant-related columns.
  • CORS is configurable through CORS_ALLOWED_ORIGINS.

Acceptance criteria

  • A documented container deployment works against managed PostgreSQL and an external inference endpoint.
  • Restarting or scaling the gateway does not lose stored conversations.
  • Codex and Claude Code can use the service through their expected credential/header conventions.
  • Conversation data is isolated by authenticated user/tenant once an identity boundary is enabled.
  • Kubernetes and Render deployment paths are both documented and smoke-tested.
  • Secrets are supplied through deployment configuration, never committed to the repository.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the remaining work in issues #107, #108, #109, and #110, then inspect the Rust server/core, existing /health and /ready endpoints, and the DATABASE_URL, CORS_ALLOWED_ORIGINS, and LLM_API_BASE configuration. The work is complete when documented Kubernetes and Render deployments use managed PostgreSQL and external inference, preserve state across restarts and replicas, honor client credentials, isolate persisted data, and pass smoke tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
kubernetes, postgresql, rust
Domain
authentication, backend, cloud, databases, infrastructure
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.