Add OpenAI-compatible inference endpoints at /api/inference/v1

Problem

dotAI's endpoints can't be driven by any standard AI client. This surfaced trying to point an agent (Vercel AI SDK) at dotCMS as its model provider:

• No tool calling anywhere in the stack. LangChain4jAIClient builds ChatRequest.builder().messages(messages).build() with no tool specifications, and AiKeys has no tools / tool_calls key. Any agent loop stops after one turn.
• No messages[]. CompletionsForm takes a single prompt string capped at 4096 chars, and /rawPrompt wraps it in one user message — no caller-supplied system prompt, no multi-turn, no assistant/tool turns.
• Streaming isn't parseable by standard clients. toSseChunk emits {"choices":[{"delta":{"content":…},"index":0}]} with no id, object, created, model, finish_reason, and no usage chunk.
• /api/v1/ai/completions is a RAG endpoint, not a chat endpoint. It runs an embeddings search, stuffs the results into the prompt, and returns a {dotCMSResults, openAiResponse} envelope.

The consequence: anyone wanting to use dotCMS as their model gateway has to write a bespoke client. Nobody does — they bypass dotCMS and put provider keys directly in their own app, which throws away the per-site credential governance dotAI already provides.

Goal

Expose the multi-provider gateway dotAI already is behind the wire format the ecosystem already speaks, so that baseURL plus a dotCMS API token is the entire integration.

We already have the expensive half: seven providers via LangChain4j (OpenAI, Azure, Bedrock, Vertex, Gemini, Anthropic, OpenRouter), per-site credentials in App secrets, model fallback chains, and connection testing. What's missing is a standard door on the front of it.

Why the OpenAI chat-completions format is the right standard: it is the de-facto wire protocol — OpenRouter, LiteLLM, Vercel AI Gateway, Ollama, vLLM, LM Studio, Groq and Together all serve it. Adopting it means dotCMS works out of the box with the Vercel AI SDK, LangChain, LlamaIndex, Cursor, n8n and the official OpenAI SDKs, with no dotCMS-specific client to write or maintain.

Endpoints

New family at /api/inference/v1. baseURL = https://<host>/api/inference/v1, dotCMS API token as the apiKey.

Endpoint	Description
POST /api/inference/v1/chat/completions	Chat completions with tool calling, multi-turn messages[], SSE streaming
POST /api/inference/v1/embeddings	Return an embedding vector for input text
GET /api/inference/v1/models	List models the site's dotAI App has configured, incl. fallback chains
POST /api/inference/v1/images/generations	Generate an image, OpenAI response shape

/api/inference/v1/completions (OpenAI's legacy non-chat completion) is deliberately out of scope — deprecated upstream.

Naming: no vendor name in the path, and deliberately not /api/ai/v1 — /api/ai/v1/embeddings and the existing /api/v1/ai/embeddings differ only by segment order and would get mistyped in configs and misread in logs. The trailing /v1 is the protocol version, intentionally distinct from dotCMS's /api/v1 resource version, so a future format (e.g. Responses API) lands at /api/inference/v2 without touching dotCMS's own versioning.

Existing /api/v1/ai/* endpoints are unaffected. text/generate, image/generate and completions/rawPrompt become superseded (soft-deprecated in docs, still functional). RAG completions, semantic search, the pgvector corpus endpoints and provider admin stay first-class — no standard verb covers them.

Site resolution

The Host header is the primary resolution path, and it is what makes baseURL alone sufficient: HTTP/1.1 requires the header, so baseURL = https://mysite.com/api/inference/v1 already carries the site with no SDK-specific configuration. No custom header is needed for the normal case.

What must not be inherited is the failure mode. HostWebAPIImpl.getCurrentHost falls through to resolveHostName(request.getServerName(), …) (HostWebAPIImpl.java:91), and HostAPIImpl.resolveHostName silently falls back to findDefaultHost when the server name matches no site or alias (HostAPIImpl.java:140-144). ConfigService.config then stacks a second silent fallback to SYSTEM_HOST when the resolved site has no dotAI secrets (ConfigService.java:49-53). A client pointed at localhost, an unaliased hostname, or sitting behind a proxy that rewrites Host would quietly spend the default (or system) site's credentials and receive a 200. GET /models answering with the system site's models instead of an empty list is the observable symptom.

Therefore, on this family:

• Resolution is strict — an unconfigured site returns an OpenAI-shaped error, never another site's providerConfig.
• An explicit override is available as the X-dotCMS-Site header (site id or hostname). A header rather than only a query param because headers are settable in every standard client (headers: {…} in the Vercel AI SDK and the OpenAI SDKs) where an extra query param is not. ?siteId= is also accepted, for parity with the existing endpoints.
• The host_id / host request parameters that HostWebAPIImpl.getCurrentHostFromRequest honours ahead of everything else (HostWebAPIImpl.java:147-166) are ignored here — a pre-existing de-facto site override that should not silently apply to a token-authenticated API.

Authentication & authorization

Any authenticated user is accepted — backend or frontend — matching the existing /api/v1/ai/* behavior, where requiredBackendUser(true).requiredFrontendUser(true) combined with the any-of role check in WebResource.checkRolePermissions (WebResource.java:428-441) admits registered frontend users. That is deliberate: a site calling AI on behalf of a visitor is a supported use case. Anonymous callers are rejected, which InitBuilder.anonAccess defaulting to AnonymousAccess.NONE already gives us (WebResource.java:932, checkAnonymousPermissions:369-385).

Because frontend users are in scope, site READ is not the load-bearing control. On any publicly delivered site, READ on the Host is necessarily granted to CMS Anonymous / frontend roles, or pages would not render — HostWebAPIImpl.checkHostPermission:110-120 runs with respectAnonPerms=true for exactly that reason. Site READ is still enforced when an explicit X-dotCMS-Site / siteId override is passed, which is a real gate against a backend user targeting a site they cannot see.

The control that actually carries weight here is model allowlisting. The OpenAI wire format lets the client name the model, so a non-admin's requested model is validated against the site's configured models rather than passed through. This carries forward the precedent in CompletionsResource.resolveForm (CompletionsResource.java:366-369), which already pins non-admins to the site's configured model, and avoids reproducing the gap in TextResource.generateRequest (TextResource.java:105,116), which passes form.model through unchecked.

Note that reading providerConfig as APILocator.systemUser() (ConfigService.java:44) is the normal dotCMS Apps design — the secret never leaves the server — so the concern is authorization to use a site's credentials, not secret leakage.

Rate limiting, cost & CORS

There is an existing backstop: RequestCostFilter is mapped at /* (web.xml:175-178) and returns 429 off LeakyTokenBucket. It is not a budget control — it is per-installation, off by default (RATE_LIMIT_ENABLED, RATE_LIMIT_MAX_BUCKET_SIZE, RATE_LIMIT_REFILL_PER_SECOND), and it prices resource-time on the local node, so it has no notion of an LLM call spending a site's money. What is in scope here is registering the cost at all (@RequestCost in the remote-HTTP-round-trip band) and mapping the upstream provider's own 429 / 5xx onto spec-shaped retryable errors, which the Vercel AI SDK needs in order to back off. A real per-site / per-token AI quota is out of scope — see the follow-ups below. Worth noting the dominant capacity risk is streaming parking a request thread for the lifetime of a completion, not request count.

CORS is deliberately not enabled. @AccessControlAllowOrigin exists (rest/annotation/AccessControlAllowOrigin.java) if it were ever wanted, but these endpoints authenticate with a long-lived dotCMS API token that can do everything its user can, and enabling CORS invites putting that token into browser JavaScript. The supported shape is server-side use — which is what every AI SDK application does anyway, via its own route handler.

Shared resolution component

Resolution and authorization belong in one component, not per resource. AiHostResolver is already the seed of this: package-private in com.dotcms.ai.rest, already carrying the strict/lenient split. It should grow into a single entry point that returns (user, host, AppConfig) with the permission check inside, and become the only way a REST resource obtains an AppConfig.

The reason is concrete: TextResource and CompletionsResource have already drifted to opposite model-passthrough policies within the same subsystem. Adding a second endpoint family with its own copy of the decision guarantees a third divergence. The existing /api/v1/ai/* resources adopt the same component in the follow-up below.

Related non-goal: no LangChain4j model construction outside LangChain4jAIClient. The model cache key is appConfig.getHost() + ":" + appConfig.getProviderConfigHash() (LangChain4jAIClient.java:147), derived from AppConfig rather than from the request, so routing through ConfigService → JSONObjectAIRequest → AIProxyClient preserves cross-site isolation for free and keeps key-rotation eviction working via flushCachesForHost (LangChain4jAIClient.java:106). Building models directly in the new resource to get tool support is what would break it.

How LangChain4j 1.15.1 supports this

Verified against langchain4j-core-1.15.1 (already pinned in bom/application/pom.xml) — every piece maps 1:1, no version bump needed:

OpenAI wire concept	LangChain4j API
tools[] in the request	ChatRequest.Builder.toolSpecifications(List<ToolSpecification>)
tool_choice	ChatRequest.Builder.toolChoice(ToolChoice)
response_format	ChatRequest.Builder.responseFormat(ResponseFormat)
tool_calls in the response	AiMessage.toolExecutionRequests() / hasToolExecutionRequests()
role: "tool" result message	ToolExecutionResultMessage(id, toolName, text)
Streaming tool-call deltas	StreamingChatResponseHandler.onPartialToolCall(PartialToolCall) / onCompleteToolCall(…)
usage	TokenUsage.inputTokenCount() / outputTokenCount() / totalTokenCount()

PartialToolCall exposes index(), id(), name() and partialArguments() — literally OpenAI's streaming tool-call delta shape, so the streaming path is closer to a field rename than a new state machine.

The work concentrates in LangChain4jAIClient (tool plumbing plus spec-conformant chunk assembly), not in the JAX-RS layer.

Acceptance Criteria

Wire format

• POST /api/inference/v1/chat/completions accepts OpenAI-shaped messages[] with system / user / assistant / tool roles and returns a spec-conformant response body
• Tool calling round-trips: tools[] in the request → tool_calls in the response → role: "tool" results accepted on the following request
• Streaming emits spec-conformant chat.completion.chunk events including id, object, created, model and finish_reason, terminated by [DONE], with tool-call arguments streamed incrementally
• GET /api/inference/v1/models lists the models configured for the resolved site, including fallback chains from ProviderConfig.allModels()
• POST /api/inference/v1/embeddings returns vectors in OpenAI response shape
• POST /api/inference/v1/images/generations returns images in OpenAI response shape

Authentication, authorization & site resolution

• Authentication works with a dotCMS API token sent as Authorization: Bearer
• Any authenticated user is accepted — backend or frontend, matching /api/v1/ai/* — and anonymous callers get 401
• The site is resolved from the HTTP Host header by default; X-dotCMS-Site (site id or hostname) and ?siteId= override it; the host_id / host request parameters are ignored on this family
• No silent site fallback: a resolved site with no dotAI configuration returns an OpenAI-shaped error rather than another site's providerConfig, and GET /models returns an empty list rather than the system site's models
• An explicit site override targeting a site the caller cannot READ returns 403
• A non-admin caller cannot name an arbitrary model: the requested model is validated against the site's configured models and an unknown one returns a NoSuchModelError-shaped 404; admins may select any model the site has configured
• Resolution and authorization live in one shared component under com.dotcms.ai.rest (grown from AiHostResolver), not duplicated per resource

Per-site isolation

• Per-site provider/credential resolution and model fallback reuse the existing AppConfig path, behaving as the existing endpoints do — with the single deliberate exception of the silent site fallbacks removed above
• Model-cache isolation holds: two sites with different provider configs under interleaved requests get separate model instances, and rotating a site's key evicts them via flushCachesForHost
• No LangChain4j model construction outside LangChain4jAIClient

Operational

• No CORS headers are emitted, and the endpoints are documented as server-side only
• Upstream provider 429 / 5xx responses map to spec-shaped retryable errors (APICallError with isRetryable) rather than leaking raw envelopes
• Each new path carries a @RequestCost price in the remote-HTTP-round-trip band (~100)
• No behavioral change to any existing /api/v1/ai/* endpoint
• End-to-end verified with the Vercel AI SDK via @ai-sdk/openai-compatible, running a multi-step tool-calling loop against a dotCMS instance configured only with baseURL + token
• openapi.yaml regenerated from the new @Operation annotations and committed

Demo Expectations

Point the Vercel AI SDK at a dotCMS instance using only baseURL and a dotCMS API token — no dotCMS-specific client code — and run a multi-step agent that calls tools, streams tokens, and completes a task. Then swap the site's dotAI provider from one vendor to another in the App config and re-run the same agent unchanged, showing dotCMS acting as the governed gateway.

Out of scope — follow-ups

• #37491 — harden the existing /api/v1/ai/* endpoints. They share the missing per-caller site check and the silent cross-site config fallback described above, and they have already drifted on model passthrough. They adopt the shared resolver there rather than here: changing authorization behavior on five shipped endpoints needs its own review, release note and rollback label, and bolting it onto this PR means neither change gets reviewed properly.
• Per-site / per-token AI spend quota. The only real answer to "an authenticated site member can spend your LLM budget," and not something the instance-wide LeakyTokenBucket can express. Needs its own design; to be filed once the shape is agreed.

Start with AiHostResolver, LangChain4jAIClient, and the existing TextResource and CompletionsResource policies described in the issue. Trace ConfigService through JSONObjectAIRequest and AIProxyClient, then review the listed LangChain4j 1.15.1 APIs and existing AI REST tests if present. Done means the four /api/inference/v1 endpoints satisfy the stated OpenAI wire-format, strict site-resolution, authorization, streaming, and error-handling criteria.