sphinx-doc / sphinx-doc/sphinx

search: rank exact matches higher than partial matches in titles

Open
#6,783 0 comments 3 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

html search type:enhancement
Dominant language
Python
Stars
8k
Forks
2.6k
PR merge metrics
No merged PRs in 30d

Description

Is your feature request related to a problem? Please describe.

In scikit-learn we are currently migrating to use sphinx search (from google search) for documentation and noticed the following limitation.

For instance, when query="sag" the first returned results is for the word "usage" rather than the word "sag".
image

Describe the solution you'd like

If I understand correctly the issues is in the scorer,
https://github.com/sphinx-doc/sphinx/blob/f7bf66012cdbf1abf75f58037752252c353e71aa/sphinx/themes/basic/static/searchtools.js#L29

where partial matches (objPartialMatch) are scored lower than full matches (objNameMatch), but because of the additive score used for object priority (objPrio), here used in a title, it ends up ranked higher. Also it looks like documentation has a higher priority than docstrings.

I think either objPrio could be decreased a bit to avoid this inversion, unless there is a reason why the were set to such high value?

Another option one could also be to use Inverse Document Frequency (IDF) as an additional weighting scheme in the scorer. "usage" is very frequent word and so mostly non relevant in any case. Though I'm not sure how well that would mix with full/partial matches and priority.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with sphinx/themes/basic/static/searchtools.js around the scorer at line 29, then trace how objPartialMatch, objNameMatch, and objPrio combine for title results. Compare the scoring behavior for a query such as "sag" and determine how exact title matches should outrank partial matches. Done means the returned ordering avoids the demonstrated inversion without undermining the documented priority behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, python
Domain
documentation, search
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.