google / google/adk-python

Evals fail with VertexAiSessionService because generated session IDs violate Agent Engine constraints

Abierto
#6,683 3 comentarios 0 reacciones 1 asignado Reclamado por @llalitkumarrr Ver en GitHub
agent engine needs review services
Lenguaje dominante
Python
Estrellas
21.5k
Forks
4k
Merge medio
1 d 14 h
PR fusionados (30 d)
37

Descripción

## 🔴 Required Information

**Describe the Bug:**

Running an evaluation with `VertexAiSessionService` fails because ADK generates evaluation session IDs using the following format:

```text
___eval___session___
```

The underscores in this ID violate the Vertex AI Agent Engine Sessions restrictions for custom session IDs. Agent Engine only accepts lowercase letters, digits, and hyphens (`[a-z0-9-]`), with a letter or digit as the first and last character.

The eval session ID is generated in `google/adk/evaluation/local_eval_service.py`:

https://github.com/google/adk-python/blob/0b55dcf9d32e22d4c8b303c3da1c275c135682bf/src/google/adk/evaluation/local_eval_service.py#L64-L68

It is then passed to the configured session service by `_get_or_create_eval_session()` in `google/adk/evaluation/evaluation_generator.py`:

https://github.com/google/adk-python/blob/0b55dcf9d32e22d4c8b303c3da1c275c135682bf/src/google/adk/evaluation/evaluation_generator.py#L118-L123

When the configured session service is `VertexAiSessionService`, Agent Engine rejects the generated ID and the evaluation cannot start.

There is also a validation mismatch in `VertexAiSessionService`. Its current local pattern accepts underscores and uppercase letters:
ref: https://docs.cloud.google.com/gemini-enterprise-agent-platform/scale/sessions/manage-with-api?hl=en#create_a_session

```python
_SESSION_ID_PATTERN = re.compile(r'^[A-Za-z0-9_-]+$')
```

Therefore, the invalid eval session ID passes ADK-side validation and is rejected only by the remote Agent Engine API.

**Steps to Reproduce:**

1. Install ADK with the required extras:

```bash
pip install "google-adk[gcp,eval]==2.6.3"
```

2. Configure an Agent Engine-backed session service:

```bash
adk web \
--session_service_uri="agentengine://projects//locations//reasoningEngines/" \
.
```

3. Open the ADK Web UI and create or select an eval set.
4. Run an evaluation.
5. Observe that creation of the generated `___eval___session___` session fails.

The problem can also be isolated through the session creation endpoint:

| Request | Result |
|---|---|
| No custom session ID | `200 OK` |
| `eval-` | `200 OK` |
| `___eval___session___` | `500 Internal Server Error` |

**Expected Behavior:**

Evaluations should be able to **create** their temporary sessions when `VertexAiSessionService` is configured.

**Observed Behavior:**

Agent Engine rejects the generated session ID:

```text
400 INVALID_ARGUMENT

session_id can only contain lowercase letters, digits and hyphens.
The first and last characters must be a letter or number.
```

The exception is subsequently exposed by the ADK Web/API server as an HTTP 500, and the evaluation fails before inference begins.

**Environment Details:**

- ADK Library Version (pip show google-adk):
- `google-adk==1.38.0`
- `google-adk==2.6.3`
- Desktop OS: macOS 26.4.1
- Python Version: Python 3.13.12

**Model Information:**

- Are you using LiteLLM: No
- Which model is being used: N/A — the failure occurs before model inference

---

## 🟡 Optional Information

**Regression:**

This is related to #2731, which reported that `VertexAiSessionService` did not support user-provided session IDs.

Current ADK releases now pass custom IDs to Agent Engine. However, evals still fail because the ADK-generated `___eval___session___` prefix does not satisfy the Agent Engine ID format.

Issue #2731 was closed as stale rather than by a merged fix, so this appears to be the remaining/current form of the compatibility problem.

**Logs:**

```text
400 INVALID_ARGUMENT. {
'error': {
'code': 400,
'message': (
'List of found errors: '
'1.Field: session_id; Message: session_id can only contain '
'lowercase letters, digits and hyphens. The first and last '
'characters must be a letter or number.'
),
'status': 'INVALID_ARGUMENT'
}
}
```

**Screenshots / Video:**

N/A

**Additional Context:**

I developed `adk eval ` version, and I found this issue.
https://github.com/ftnext/pytest-adk/releases/tag/0.0.7

**Minimal Reproduction Code:**

```python
import asyncio

from google.adk.sessions import VertexAiSessionService

async def main() -> None:
service = VertexAiSessionService(
project="",
location="",
agent_engine_id="",
)

await service.create_session(
app_name="",
user_id="test-user",
session_id="___eval___session___550e8400-e29b-41d4-a716-446655440000",
)

asyncio.run(main())
```

**How often has this issue occurred?:**

- Always (100%)

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.