microsoft / microsoft/azurechat
Token limit reached in file search.
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 1.4k
- Forks
- 1.3k
- Avg merge
- 1d 11h
- Merged PRs (30d)
- 1
Description
everyone
I uploaded the file and started the chat and AOAI returned an error saying that the token limit has been reached.
I have taken care of this by changing the search limit from 10 to 5 as a simple solution,
However, since this did not occur with the previous UI, we thought there might be another problem and investigated, and discovered that the chunk size was different.
Currently the upper limit is 2300, but previously it was 1000.
We believe that although the chunk size has changed, the search limit has not, and therefore the tokens sent have reached the limit.
Note that we are using gpt-3.5-16k as the model.
We believe that individual users will modify this according to their situation, but we would like to inform you that this phenomenon has occurred.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No file, test, or entry point is named. Start by reproducing file search with the reported 2300 chunk-size limit, search limit of 10, and gpt-3.5-16k model; trace how chunks and search results are sent to Azure OpenAI. Done means the token-limit behavior and the required limit relationship are verified and covered by an appropriate regression test or documented configuration.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- azure, typescript
- Domain
- ai, search
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100