google / google/adk-java

Feature: Support third-party model strings like groq/model-id via a ServiceLoader SPI

Abierto
#1,362 1 comentario 0 reacciones 1 asignado Reclamado por @hemasekhar-p Ver en GitHub
needs review
Lenguaje dominante
Java
Estrellas
1.7k
Forks
420
Merge medio
4 d 12 h
PR fusionados (30 d)
31

Descripción

**Please make sure you read the contribution guide and file the issues in the right place.**
[Contribution guide.](https://google.github.io/adk-docs/contributing-guide/)

## 🔴 Required Information

### Is your feature request related to a specific problem?

Yes. ADK Java has no convention-based way to resolve a model name string to a third-party
backend. The official bridges (`contrib/langchain4j`, `contrib/spring-ai`) cover many
providers, but agents receive constructed model *objects* — the string form
(`.model("groq/")`) does not resolve through them. The only native mechanism for
string resolution today is calling `LlmRegistry.registerLlm(pattern, factory)` manually in
every application before any agent is built.

Python ADK gets zero-code provider switching via LiteLLM — a model string from config resolves
at runtime. Java has no equivalent, which hurts:

- **Multi-agent cost optimization** — assigning different models to different agents in one
pipeline (Gemini where its built-in tools are needed, fast hosted models for generation,
local Ollama for lightweight steps) currently requires provider construction code in every
application.
- **Configuration-driven deployments** — the model cannot be swapped via config alone; a code
change and rebuild are required.
- **Local development** — using Ollama offline should be a dependency swap, not code.

There is also a subtle correctness constraint any solution must respect. When an agent is
created with `.model("groq/some-model")`, that string is not what reaches the provider:
`LlmRegistry` calls the registered factory with the requested name, and the `"model"` field in
the outgoing request is whatever name the *returned `BaseLlm` was constructed with*
(`Basic.java` copies it into `LlmRequest.model`; `ChatCompletionsRequest` sends it verbatim).
Since providers accept only bare model IDs, pattern-based registration needs a **name-aware
factory**: build a distinct instance per requested name and strip the routing prefix before
construction, so the backend receives an ID it recognizes rather than `"groq/some-model"`.

Related: [#1198](https://github.com/google/adk-java/issues/1198), [#1202](https://github.com/google/adk-java/pull/1202), [#350](https://github.com/google/adk-java/discussions/350).

### Describe the Solution You'd Like

A minimal SPI in ADK core (3 classes, no new dependencies), discovered via the standard
`java.util.ServiceLoader` mechanism:

- **`ModelProvider`** — SPI interface: `prefix()` (routing namespace, e.g. `groq`) +
`createFromBareModelName(name)` → `BaseLlm`. A default `create(name)` strips the prefix, so
the backend always receives the bare model ID.
- **`ModelProviderRegistry`** — `registerAll()` discovers all `ModelProvider` implementations
on the classpath via `ServiceLoader` and registers each with the existing `LlmRegistry`.
Providers that fail to instantiate are logged and skipped without affecting the rest.
- **`OpenAiCompatibleLlm`** — a thin `BaseLlm` over ADK's native
`ChatCompletionsHttpClient` for any `POST /v1/chat/completions` endpoint (Groq, Ollama,
OpenRouter, ...). Constructed with the bare model ID, which is what the
backend receives as the wire-format `"model"` field.

Usage — one explicit opt-in line, then model strings resolve from the classpath:

```java
ModelProviderRegistry.registerAll();

LlmAgent agent =
LlmAgent.builder()
.name("assistant")
.model("groq/some-model") // resolved via the Groq provider JAR
.build();
```

A provider is a one-class JAR plus a `META-INF/services/com.google.adk.models.ModelProvider`
entry — no ADK changes needed to add new providers, ever:

```java
public final class GroqModelProvider implements ModelProvider {
@Override public String prefix() { return "groq"; }

@Override public BaseLlm createFromBareModelName(String bareModelName) {
return new OpenAiCompatibleLlm(
bareModelName,
"https://api.groq.com/openai/v1",
Optional.ofNullable(System.getenv("GROQ_API_KEY")));
}
}
```

**Deliberately out of scope** (to keep the change small and uncontroversial): bundled provider
implementations, demo modules, and automatic `registerAll()` inside `Runner`/`AdkWebServer`
(explicit opt-in first; auto-registration can be a follow-up discussion).

### Impact on your work

**Impact Level:** Medium

Not a hard blocker — everything here can be done today with manual `LlmRegistry.registerLlm`
calls. The cost is recurring friction: every ADK Java application re-implements the same
registration and prefix-stripping boilerplate to mix models per agent (Gemini where its
built-in tools are needed, hosted models like Groq/OpenRouter for generation, local Ollama for
lightweight steps).

The main benefit is to the ADK Java ecosystem: convention-based model selection lets Java
developers build multi-agent systems on any mix of hosted, free-tier, and local models by
changing only dependencies and configuration — no provider-specific Java code in the
application — matching the provider switching Python ADK users already get via LiteLLM and
closing a gap between the two SDKs.

### Willingness to contribute

**Yes** — the implementation is complete and ready to submit:

- ✅ 3 core classes as described (no new dependencies; native `ChatCompletionsHttpClient` path)
- ✅ 18 unit tests, all passing (`mvn -pl core test`), including an end-to-end test that a
registered provider resolves `groq/`-style strings to an LLM carrying the bare model ID
- ✅ google-java-format applied; follows existing `com.google.adk.models` conventions
- ✅ Verified end-to-end against live endpoints (Groq, OpenRouter) with external provider
JARs discovered via `ServiceLoader`, including streaming and tool calling

**Can submit the PR immediately.**

---

## 🟡 Recommended Information

### Describe Alternatives You've Considered

1. **The official bridges — `contrib/langchain4j` and `contrib/spring-ai`** (status quo) —
both solve implementation boilerplate but not discovery: agents receive a
constructed/injected model *object*, not a resolvable name string. This proposal addresses
the discovery gap for OpenAI-compatible endpoints; the bridges remain the path for other
providers.
2. **Instance-based registration** (e.g. the builder + `registerWithPattern` convenience
sketched in [#1202](https://github.com/google/adk-java/pull/1202)) — a pre-built model object is registered directly. This proposal
registers *factories* instead: `LlmRegistry` passes each requested name to the factory,
which strips the prefix and constructs an instance configured for exactly that model — so
any number of models can be used under one prefix (e.g. `groq/llama-3.3-70b-versatile` and
`groq/gemma2-9b-it` in the same pipeline) from a single registration. Happy to align the
two efforts in whichever direction maintainers prefer.
3. **Manual `LlmRegistry.registerLlm` calls in each application** — ADK's only native mechanism
for string resolution today. It works, but it is exactly the boilerplate this proposal
removes, and each application re-implements prefix stripping (or forgets to).

### Additional Context

The native `ChatCompletionsHttpClient` path became fully viable for strict OpenAI-compatible
providers in ADK 1.6.0, when the JSON Schema serialization fixes landed ([#1265](https://github.com/google/adk-java/issues/1265)/[#1266](https://github.com/google/adk-java/pull/1266)).

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.