googleapis / googleapis/python-genai

Vertex AI: service_tier in the request body is accepted and silently ignored (re-file of #2433)

Open
#2,847 1 comment 0 reactions 1 assignee Claimed by @Venkaiahbabuneelam View on GitHub
priority: p2 type: bug
Dominant language
Python
Stars
4k
Forks
1k
Avg merge
2d 11h
Merged PRs (30d)
40

Description

**Environment details**

- Programming language: Python
- OS: macOS 26.5
- Language runtime version: 3.11
- Package version: `google-genai` 2.17.0

**Issue**

Re-filing #2433, which the stale bot closed on 2026-06-02 without a fix. Four people reported it again afterwards on 2.14.0, 2.16.0 and 1.73.1, and the bot's closing message asks for a new issue, so here is one with measurements.

On the Vertex AI backend, `GenerateContentConfig.service_tier` is serialized into the request body (`_GenerateContentConfig_to_vertex` sets `serviceTier`). Vertex does not read the tier from the body. The result is not an error, which is the problem: the request succeeds and bills the standard tier.

Measured today, project on `global`, `gemini-3.5-flash`, same project and model in every row:

| What was sent | HTTP | `usageMetadata.trafficType` |
|---|---|---|
| nothing (baseline) | 200 | `ON_DEMAND` |
| body `serviceTier: 'flex'` (the `ServiceTier.FLEX` value) | **400** `Invalid value at 'service_tier'` | — |
| body `serviceTier: 'SERVICE_TIER_FLEX'` | 200 | `ON_DEMAND` |
| header `X-Vertex-AI-LLM-Shared-Request-Type: flex` | 200 | **`ON_DEMAND_FLEX`** |
| header `X-Vertex-AI-LLM-Shared-Request-Type: priority` | 200 | **`ON_DEMAND_PRIORITY`** |

Two distinct problems in there:

1. The values in the `ServiceTier` enum (`flex`, `priority`, `standard`) are the Gemini Developer API spellings. Vertex rejects them and wants `SERVICE_TIER_FLEX`. So `service_tier=ServiceTier.FLEX` with `vertexai=True` is a guaranteed `400`.
2. Using the spelling Vertex does accept gets a `200` and standard billing. Silent, and a 2x price difference against `priority`. One user on #2433 wrote "we thought we were getting billed as flex but I'm worried we might be paying 2x without knowing".

In #2433 a maintainer said on 2026-05-21 that "the Vertex backend will soon support ServiceTier in the payload". As of 2.17.0 and today's measurement it does not, and there is still no error to tell you so.

**Repro**

```python
from google import genai
from google.genai import types

client = genai.Client(vertexai=True, project="YOUR_PROJECT", location="global")

for sent in [None, "SERVICE_TIER_FLEX"]:
config = types.GenerateContentConfig(service_tier=sent) if sent else types.GenerateContentConfig()
response = client.models.generate_content(
model="gemini-3.5-flash", contents="hi", config=config
)
print(sent, "->", response.usage_metadata.traffic_type)
# None -> TrafficType.ON_DEMAND
# SERVICE_TIER_FLEX -> TrafficType.ON_DEMAND <- requested flex, billed standard
```

Swapping the body field for the header returns `ON_DEMAND_FLEX` on the same project and model, so nothing about the account or the model is in the way.

Note that flex is per model, not only per project: it is served on `gemini-3.1-pro-preview`, `gemini-3.5-flash` and `gemini-3.1-flash-lite`, while the 2.5 family returns `400 Flex API is not supported for model: ...`. That `400` only appears on the header path. On the body path an unsupported model is indistinguishable from a supported one.

**Related**

- #2445 already implements the header translation and has been open since 2026-05-15.
- #2435 covers the response side, `usageMetadata.serviceTier` missing from `GenerateContentResponseUsageMetadata`. Confirmed on 2.17.0: the Developer API returns it, the SDK model has no such attribute, so the effective tier is not observable from the parsed response.
- Same mismatch filed against the other SDKs: googleapis/java-genai#972 and googleapis/js-genai#1468.

Until an SDK release covers this, anything reading the tier from the body on Vertex is silently paying standard. Happy to help test a fix.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.