microsoft / microsoft/azurechat

Token limit reached in file search.

Open
#367 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
1.4k
Forks
1.3k
Avg merge
1d 11h
Merged PRs (30d)
1

Description

everyone

I uploaded the file and started the chat and AOAI returned an error saying that the token limit has been reached.

I have taken care of this by changing the search limit from 10 to 5 as a simple solution,
However, since this did not occur with the previous UI, we thought there might be another problem and investigated, and discovered that the chunk size was different.

Currently the upper limit is 2300, but previously it was 1000.

We believe that although the chunk size has changed, the search limit has not, and therefore the tokens sent have reached the limit.

Note that we are using gpt-3.5-16k as the model.

We believe that individual users will modify this according to their situation, but we would like to inform you that this phenomenon has occurred.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No file, test, or entry point is named. Start by reproducing file search with the reported 2300 chunk-size limit, search limit of 10, and gpt-3.5-16k model; trace how chunks and search results are sent to Azure OpenAI. Done means the token-limit behavior and the required limit relationship are verified and covered by an appropriate regression test or documented configuration.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure, typescript
Domain
ai, search
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.