internetarchive / internetarchive/openlibrary
Create dataset of examples of poor search edition ranking
- Dominant language
- Python
- Stars
- 6.7k
- Forks
- 2k
- Avg merge
- 2d 19h
- Merged PRs (30d)
- 138
Description
Edition ranking is what edition (title, cover) is chosen for a given work in search results. E.g. is it the french edition? One with a cover? One that is readable? This issue is complete once we have a list of ~20-50 cases where the _wrong_ edition was selected for a certain query.
The current logic is as follows:
https://github.com/internetarchive/openlibrary/blob/09cbf7d9b2868abdf7abfbf9b0746751a747207d/openlibrary/plugins/worksearch/schemes/works.py#L522-L539
We have noticed various cases where this ranking appears to not be operating correctly. We hypothesize some reasons could be:
* the inclusion of stopwords in our index causing "the lord of the rings" and "lord of the rings" to rank differently
* the term frequency algorithm our solr is configured to use, ala https://github.com/internetarchive/openlibrary/issues/10969#issuecomment-3096849420
* `language`, other key metadata missing from the work records
In order to test improvements, we need a list of failure cases we can use to adjust the ranking algorithm, and then compare a before/after. We can build a control set by using validate popular book queries, common queries from our logs, and random queries from our logs (like in our previous [search evaluation](https://docs.google.com/spreadsheets/d/1BN5I7-OkTPaoTr2Es6jQ4O9ICWFmH0q9CP6kEgolCgg/edit#gid=1006480604)). But finding specific failure cases automatically is tricky, so we will collect failure cases on the issue.
### Stakeholders
@mekarpeles
Contributor guide
Research direction
Read the edition-ranking logic in openlibrary/plugins/worksearch/schemes/works.py at lines 522-539, then review the linked search-evaluation spreadsheet and the suggested popular, logged, and random query sources. Collect approximately 20-50 cases where the wrong edition is selected for a query, recording each failure clearly so the set can be used to compare ranking changes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- search
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100