Bm25: Expanding keywords & custom vector space
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 3.2k
- Forks
- 248
- Avg merge
- 4d 8h
- Merged PRs (30d)
- 4
Description
What happened?
Is it possible to expand keywords via the current BM25 impelemtation?
Or even better, build the bm25 vector space on the documents?
Very often the corpus of documents is pre-determined and largely static, so the custom vector space & keyword extension (maybe via dense vectors?) would potentially improve BM25 performance
What Python version are you on? e.g. python --version
Python3.11 with pyenv
Version
0.2.7 (Latest)
What os are you seeing the problem on?
No response
Relevant stack traces and/or logs
No response
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating the current BM25 implementation and reviewing how it builds its keyword space from documents. The request raises several possible directions—keyword expansion, a corpus-derived vector space, or dense vectors—so clarify the intended scope before implementation. Done should include a defined approach and tests demonstrating improved or extended BM25 behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- search
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100