Feature: Support third-party model strings like groq/model-id via a ServiceLoader SPI
- Lingua principale
- Java
- Stelle
- 1.7k
- Fork
- 420
- Merge medio
- 4g 12h
- PR unite (30g)
- 31
Descrizione
**Please make sure you read the contribution guide and file the issues in the right place.**
[Contribution guide.](https://google.github.io/adk-docs/contributing-guide/)
## 🔴 Required Information
### Is your feature request related to a specific problem?
Yes. ADK Java has no convention-based way to resolve a model name string to a third-party
backend. The official bridges (`contrib/langchain4j`, `contrib/spring-ai`) cover many
providers, but agents receive constructed model *objects* — the string form
(`.model("groq/")`) does not resolve through them. The only native mechanism for
string resolution today is calling `LlmRegistry.registerLlm(pattern, factory)` manually in
every application before any agent is built.
Python ADK gets zero-code provider switching via LiteLLM — a model string from config resolves
at runtime. Java has no equivalent, which hurts:
- **Multi-agent cost optimization** — assigning different models to different agents in one
pipeline (Gemini where its built-in tools are needed, fast hosted models for generation,
local Ollama for lightweight steps) currently requires provider construction code in every
application.
- **Configuration-driven deployments** — the model cannot be swapped via config alone; a code
change and rebuild are required.
- **Local development** — using Ollama offline should be a dependency swap, not code.
There is also a subtle correctness constraint any solution must respect. When an agent is
created with `.model("groq/some-model")`, that string is not what reaches the provider:
`LlmRegistry` calls the registered factory with the requested name, and the `"model"` field in
the outgoing request is whatever name the *returned `BaseLlm` was constructed with*
(`Basic.java` copies it into `LlmRequest.model`; `ChatCompletionsRequest` sends it verbatim).
Since providers accept only bare model IDs, pattern-based registration needs a **name-aware
factory**: build a distinct instance per requested name and strip the routing prefix before
construction, so the backend receives an ID it recognizes rather than `"groq/some-model"`.
Related: [#1198](https://github.com/google/adk-java/issues/1198), [#1202](https://github.com/google/adk-java/pull/1202), [#350](https://github.com/google/adk-java/discussions/350).
### Describe the Solution You'd Like
A minimal SPI in ADK core (3 classes, no new dependencies), discovered via the standard
`java.util.ServiceLoader` mechanism:
- **`ModelProvider`** — SPI interface: `prefix()` (routing namespace, e.g. `groq`) +
`createFromBareModelName(name)` → `BaseLlm`. A default `create(name)` strips the prefix, so
the backend always receives the bare model ID.
- **`ModelProviderRegistry`** — `registerAll()` discovers all `ModelProvider` implementations
on the classpath via `ServiceLoader` and registers each with the existing `LlmRegistry`.
Providers that fail to instantiate are logged and skipped without affecting the rest.
- **`OpenAiCompatibleLlm`** — a thin `BaseLlm` over ADK's native
`ChatCompletionsHttpClient` for any `POST /v1/chat/completions` endpoint (Groq, Ollama,
OpenRouter, ...). Constructed with the bare model ID, which is what the
backend receives as the wire-format `"model"` field.
Usage — one explicit opt-in line, then model strings resolve from the classpath:
```java
ModelProviderRegistry.registerAll();
LlmAgent agent =
LlmAgent.builder()
.name("assistant")
.model("groq/some-model") // resolved via the Groq provider JAR
.build();
```
A provider is a one-class JAR plus a `META-INF/services/com.google.adk.models.ModelProvider`
entry — no ADK changes needed to add new providers, ever:
```java
public final class GroqModelProvider implements ModelProvider {
@Override public String prefix() { return "groq"; }
@Override public BaseLlm createFromBareModelName(String bareModelName) {
return new OpenAiCompatibleLlm(
bareModelName,
"https://api.groq.com/openai/v1",
Optional.ofNullable(System.getenv("GROQ_API_KEY")));
}
}
```
**Deliberately out of scope** (to keep the change small and uncontroversial): bundled provider
implementations, demo modules, and automatic `registerAll()` inside `Runner`/`AdkWebServer`
(explicit opt-in first; auto-registration can be a follow-up discussion).
### Impact on your work
**Impact Level:** Medium
Not a hard blocker — everything here can be done today with manual `LlmRegistry.registerLlm`
calls. The cost is recurring friction: every ADK Java application re-implements the same
registration and prefix-stripping boilerplate to mix models per agent (Gemini where its
built-in tools are needed, hosted models like Groq/OpenRouter for generation, local Ollama for
lightweight steps).
The main benefit is to the ADK Java ecosystem: convention-based model selection lets Java
developers build multi-agent systems on any mix of hosted, free-tier, and local models by
changing only dependencies and configuration — no provider-specific Java code in the
application — matching the provider switching Python ADK users already get via LiteLLM and
closing a gap between the two SDKs.
### Willingness to contribute
**Yes** — the implementation is complete and ready to submit:
- ✅ 3 core classes as described (no new dependencies; native `ChatCompletionsHttpClient` path)
- ✅ 18 unit tests, all passing (`mvn -pl core test`), including an end-to-end test that a
registered provider resolves `groq/`-style strings to an LLM carrying the bare model ID
- ✅ google-java-format applied; follows existing `com.google.adk.models` conventions
- ✅ Verified end-to-end against live endpoints (Groq, OpenRouter) with external provider
JARs discovered via `ServiceLoader`, including streaming and tool calling
**Can submit the PR immediately.**
---
## 🟡 Recommended Information
### Describe Alternatives You've Considered
1. **The official bridges — `contrib/langchain4j` and `contrib/spring-ai`** (status quo) —
both solve implementation boilerplate but not discovery: agents receive a
constructed/injected model *object*, not a resolvable name string. This proposal addresses
the discovery gap for OpenAI-compatible endpoints; the bridges remain the path for other
providers.
2. **Instance-based registration** (e.g. the builder + `registerWithPattern` convenience
sketched in [#1202](https://github.com/google/adk-java/pull/1202)) — a pre-built model object is registered directly. This proposal
registers *factories* instead: `LlmRegistry` passes each requested name to the factory,
which strips the prefix and constructs an instance configured for exactly that model — so
any number of models can be used under one prefix (e.g. `groq/llama-3.3-70b-versatile` and
`groq/gemma2-9b-it` in the same pipeline) from a single registration. Happy to align the
two efforts in whichever direction maintainers prefer.
3. **Manual `LlmRegistry.registerLlm` calls in each application** — ADK's only native mechanism
for string resolution today. It works, but it is exactly the boilerplate this proposal
removes, and each application re-implements prefix stripping (or forgets to).
### Additional Context
The native `ChatCompletionsHttpClient` path became fully viable for strict OpenAI-compatible
providers in ADK 1.6.0, when the JSON Schema serialization fixes landed ([#1265](https://github.com/google/adk-java/issues/1265)/[#1266](https://github.com/google/adk-java/pull/1266)).
Guida per i contributori
Apri la guida per i contributori
Valutazione
Questa issue non è ancora stata valutata.