posit-dev / posit-dev/positron
Assistant: Send console history entries most relevant to query, rather than most recent
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 4.3k
- Forks
- 184
- Avg merge
- 1d 9h
- Merged PRs (30d)
- 195
Description
(via a tech demo by @georgestagg)
Currently, Positron Assistant sends the most recent console history entries to the model as context. Entries are prioritized based only on their recency, so we just start with the newest entries and keep going until we run out of budget.
It is true that the most recent entries are overall more likely to be relevant than older entries, but it shouldn't be the only factor. In particular, the user may be asking about something other than their most recent query, and it is likely that code run further in the history is important as context to the question.
To improve Assistant's ability to use console history as context without stuffing the whole history into the context, we should consider using something like BM25 to help select the most relevant console entries.
(We could also use something like this when selecting variables, if there are a large number of them.)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at src/vs/workbench/contrib/chat/browser/contrib/chatRuntimeSessionContext.ts#L346-L360, where recent console history entries are selected for Assistant context. Review how the current recency-based selection works and investigate BM25 or a similar relevance-ranking approach. Done should mean relevant console entries are selected within the context budget, with behavior validated by appropriate tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- ai, developer-experience
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100