openai / openai/openai-openapi
Token Limit for gpt-4o-mini Model in v1/chat/completions API
Nobody has claimed this yet.
- Dominant language
- No language data
- Stars
- 2.5k
- Forks
- 527
- Avg merge
- 1h 46m
- Merged PRs (30d)
- 2
Description
I have been testing the gpt-4o-mini model using the v1/chat/completions API and have encountered an issue regarding the token limit. According to the official documentation, the context window for the gpt-4o-mini model is specified as 128,000 tokens. However, in my tests, the total token length limit seems to be restricted to approximately 16,000 tokens instead.
Here are the details of my test:
Model: gpt-4o-mini
Test Scenario: I included a history conversation of around 16,000 tokens and set max_tokens to 4,000.
Issue: The response was truncated, and I only received a few hundred tokens of output. model in response is gpt-4o-mini-2024-07-18. The stop_reason: length in the response indicates that the length limit was exceeded, despite the remaining difference between 16,000 tokens and the documented 128,000 tokens being substantial.
I would appreciate it if you could investigate this discrepancy and confirm whether there are any additional limitations or if this might be an issue with the API. Your assistance in clarifying this matter would be greatly valued.
Thank you.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the gpt-4o-mini request through the v1/chat/completions API with the reported conversation size and max_tokens value. Compare the documented 128,000-token context window with the response model, stop_reason, and observed output limit; done means confirming the limitation or identifying the discrepancy and documenting the explanation.
Written by the indexing model from the issue text.
Assessment
- Domain
- api
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100