[Bug]: Token-based rate limit charges 1 token for requests with no token usage

Please select the area the issue is related to

Gateway

Please select the aspect the issue is related to

Aspect/API (API backends, definitions, contracts, interfaces, OpenAPI)

Description

When a token-based-ratelimit policy is attached to an LlmProvider via globalPolicies, it applies to every request within the provider's context. If the response does not contain the fields defined by the provider template for extracting token usage, the policy does not treat the request as consuming 0 tokens. Instead, it falls back to charging 1 token against the configured token quota.

This causes non-token-consuming endpoints to incorrectly reduce the token budget.

For example, an Anthropic provider template extracts token usage from:

$.usage.input_tokens and $.usage.output_tokens

However, endpoints such as GET /v1/models do not return a usage object:

Since neither input_tokens nor output_tokens can be extracted, the request should consume 0 tokens. Instead, the token-based rate limit charges 1 token.

After 75 requests to a non-token-consuming endpoint, the entire 75-token budget is exhausted, even though no LLM inference has occurred.

The same issue can occur with other management endpoints that do not return token usage.

Additional Concern

This also raises a concern about the scope of the token quota. If a token-consuming resource exhausts the configured token budget and gets throttled, will non-token-consuming resources under the same provider context also be throttled?

For example, if the token budget is exhausted by actual LLM inference requests, a request such as GET /v1/models, which does not consume any tokens, should ideally remain accessible rather than being throttled due to the exhausted token quota.

Steps to Reproduce

1. Create an Anthropic LlmProvider and configure a token-based-ratelimit policy with a token quota of 5.

2. Send 5 requests to the provider's model-listing endpoint.

3. Observe that each request consumes 1 token from the token quota, even though the endpoint does not consume any LLM tokens.

4. Send a 6th request to the same endpoint.

5. Observe that the 6th request is throttled because the token quota has been exhausted.

6. This demonstrates that requests to a non-token-consuming resource incorrectly consume the token quota and can eventually cause the resource itself to be throttled.

Severity Level of the Issue

Severity/Blocker (Core functionality is broken and consumer is blocked. Need immediate attention)

Environment Details (with versions)

No response

Start by tracing the token-based-ratelimit policy attached through globalPolicies and how the Anthropic provider template extracts usage from $.usage.input_tokens and $.usage.output_tokens. Reproduce the behavior with the provider's GET /v1/models endpoint and a five-token quota. Done means requests with no usage consume 0 tokens, and the scope of throttling for non-token-consuming resources is documented or resolved.