huggingface / huggingface/agents-course

[UPDATE] The Use of BM25 for Semantic Search Example in Unit 2.1 retrieval_agents.mdx

Open
#319 4 comments 2 reactions 0 assignees View on GitHub
documentation
Dominant language
MDX
Stars
32.6k
Forks
2.3k
PR merge metrics
No merged PRs in 30d

Description

The `PartyPlanningRetrieverTool` subclass in the example retrieval code currently uses BM25 (`BM25Retriever`), which I believe is a lexical search algorithm. However, the description labels this as a "semantic search," which may cause confusion. Unlike vector-based retrieval methods, BM25 does not capture the meaning or context of words but instead relies on keyword frequency and ranking.
This builds on the clarification from my previous PR (# 279 merged and closed), which revised the definition of vector databases and semantic search to avoid similar confusion.

Proposed Fix:
Option 1: Clearly state that this code example uses a keyword retrieval method, which is BM25, rather than a semantic search.
Option 2: Replace BM25 with a vector-based retriever to demonstrate a true semantic search approach.

Let me know what you think and if you'd like any help.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.