internetarchive / internetarchive/openlibrary
Tweak solr cache configs
- Dominant language
- Python
- Stars
- 6.7k
- Forks
- 2k
- Avg merge
- 2d 19h
- Merged PRs (30d)
- 138
Description
We currently have two caches that have very high hit rates, the `filterCache` and the `perSegFilter` cache
`filterCache` is I believe used whenever with use the `fq` parameter. We use this a lot of things like filtered author queries, reading log queries, etc.
`perSegFilter` is apparently "custom cache currently used by block join". I believe we use block join indirectly for our edition-aware queries, which power huge chunks of the site.
Increasing the sizes of these caches might improve solr performance.
### Stakeholders
@scottbarnes @tfmorris
Contributor guide
Research direction
Start by locating the Solr configuration entries for the filterCache and perSegFilter caches, then review the linked cache hit-rate evidence and their current sizes. Compare potential size increases against observed Solr performance and define completion as a validated configuration change that improves performance without unacceptable resource use.
Written by the indexing model from the issue text.
Assessment
- Domain
- backend, performance, search
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100