google / google/adk-python

Evals fail with VertexAiSessionService because generated session IDs violate Agent Engine constraints

Aperta
#6,683 3 commenti 0 reazioni 1 assegnatario Rivendicata da @llalitkumarrr Vedi su GitHub
agent engine needs review services
Lingua principale
Python
Stelle
21.5k
Fork
4k
Merge medio
1g 14h
PR unite (30g)
37

Descrizione

## 🔴 Required Information

**Describe the Bug:**

Running an evaluation with `VertexAiSessionService` fails because ADK generates evaluation session IDs using the following format:

```text
___eval___session___
```

The underscores in this ID violate the Vertex AI Agent Engine Sessions restrictions for custom session IDs. Agent Engine only accepts lowercase letters, digits, and hyphens (`[a-z0-9-]`), with a letter or digit as the first and last character.

The eval session ID is generated in `google/adk/evaluation/local_eval_service.py`:

https://github.com/google/adk-python/blob/0b55dcf9d32e22d4c8b303c3da1c275c135682bf/src/google/adk/evaluation/local_eval_service.py#L64-L68

It is then passed to the configured session service by `_get_or_create_eval_session()` in `google/adk/evaluation/evaluation_generator.py`:

https://github.com/google/adk-python/blob/0b55dcf9d32e22d4c8b303c3da1c275c135682bf/src/google/adk/evaluation/evaluation_generator.py#L118-L123

When the configured session service is `VertexAiSessionService`, Agent Engine rejects the generated ID and the evaluation cannot start.

There is also a validation mismatch in `VertexAiSessionService`. Its current local pattern accepts underscores and uppercase letters:
ref: https://docs.cloud.google.com/gemini-enterprise-agent-platform/scale/sessions/manage-with-api?hl=en#create_a_session

```python
_SESSION_ID_PATTERN = re.compile(r'^[A-Za-z0-9_-]+$')
```

Therefore, the invalid eval session ID passes ADK-side validation and is rejected only by the remote Agent Engine API.

**Steps to Reproduce:**

1. Install ADK with the required extras:

```bash
pip install "google-adk[gcp,eval]==2.6.3"
```

2. Configure an Agent Engine-backed session service:

```bash
adk web \
--session_service_uri="agentengine://projects//locations//reasoningEngines/" \
.
```

3. Open the ADK Web UI and create or select an eval set.
4. Run an evaluation.
5. Observe that creation of the generated `___eval___session___` session fails.

The problem can also be isolated through the session creation endpoint:

| Request | Result |
|---|---|
| No custom session ID | `200 OK` |
| `eval-` | `200 OK` |
| `___eval___session___` | `500 Internal Server Error` |

**Expected Behavior:**

Evaluations should be able to **create** their temporary sessions when `VertexAiSessionService` is configured.

**Observed Behavior:**

Agent Engine rejects the generated session ID:

```text
400 INVALID_ARGUMENT

session_id can only contain lowercase letters, digits and hyphens.
The first and last characters must be a letter or number.
```

The exception is subsequently exposed by the ADK Web/API server as an HTTP 500, and the evaluation fails before inference begins.

**Environment Details:**

- ADK Library Version (pip show google-adk):
- `google-adk==1.38.0`
- `google-adk==2.6.3`
- Desktop OS: macOS 26.4.1
- Python Version: Python 3.13.12

**Model Information:**

- Are you using LiteLLM: No
- Which model is being used: N/A — the failure occurs before model inference

---

## 🟡 Optional Information

**Regression:**

This is related to #2731, which reported that `VertexAiSessionService` did not support user-provided session IDs.

Current ADK releases now pass custom IDs to Agent Engine. However, evals still fail because the ADK-generated `___eval___session___` prefix does not satisfy the Agent Engine ID format.

Issue #2731 was closed as stale rather than by a merged fix, so this appears to be the remaining/current form of the compatibility problem.

**Logs:**

```text
400 INVALID_ARGUMENT. {
'error': {
'code': 400,
'message': (
'List of found errors: '
'1.Field: session_id; Message: session_id can only contain '
'lowercase letters, digits and hyphens. The first and last '
'characters must be a letter or number.'
),
'status': 'INVALID_ARGUMENT'
}
}
```

**Screenshots / Video:**

N/A

**Additional Context:**

I developed `adk eval ` version, and I found this issue.
https://github.com/ftnext/pytest-adk/releases/tag/0.0.7

**Minimal Reproduction Code:**

```python
import asyncio

from google.adk.sessions import VertexAiSessionService

async def main() -> None:
service = VertexAiSessionService(
project="",
location="",
agent_engine_id="",
)

await service.create_session(
app_name="",
user_id="test-user",
session_id="___eval___session___550e8400-e29b-41d4-a716-446655440000",
)

asyncio.run(main())
```

**How often has this issue occurred?:**

- Always (100%)

Guida per i contributori

Apri la guida per i contributori

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.