Azure / Azure/azure-rest-api-specs

Open AI chat completion API "Retry-After" header is not exposed to the browser

Open
#24,904 4 comments 2 reactions 0 assignees View on GitHub
customer-reported needs-team-attention question Service Attention
Dominant language
TypeSpec
Stars
3.1k
Forks
5.9k
Avg merge
3d 2h
Merged PRs (30d)
424

Description

Hi, I'm trying to use stay within the tokens per minute limited enforced by the Open AI Chat Completion API.

When server responded with 429 error, I'm expecting the `Retry-After` field in the header exposed to the browser so the caller can retry after the server suggested period of time. The [documentation here](https://learn.microsoft.com/en-us/azure/ai-services/openai/quotas-limits#general-best-practices-to-remain-within-rate-limits) specially calls for such retry strategy

> General best practices to remain within rate limits
>
> To minimize issues related to rate limits, it's a good idea to use the following techniques:
> - Implement retry logic in your application.
> - ...

The problem is that the response header is set such that the `Retry-After` header field is hidden from the browser. See screenshot below:

![image](https://github.com/Azure/azure-rest-api-specs/assets/1895289/5780ab8a-f1f2-48ad-91af-07088967025e)

This makes it difficult for the client to set the right timeout. Please consider adding `Retry-After` to the `Access-Control-Expose-Headers` field.

References:
- [Understanding rate limit](https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/quota#understanding-rate-limits)
- [General best practices to remain within rate limits](https://learn.microsoft.com/en-us/azure/ai-services/openai/quotas-limits)

Contributor guide

Open the contributing guide

Research direction

No file or test is named in the issue. Start by locating the OpenAI Chat Completion API specification and its response-header configuration, then verify the browser-visible headers for a 429 response; done means Retry-After is included in Access-Control-Expose-Headers.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure, openapi
Domain
api
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.