apache / apache/lucene

Switch to a "double barrel" HPPC cache for the taxonomy LRU cache [LUCENE-10068]

Open
#11,106 8 comments 0 reactions 0 assignees View on GitHub
affects-version:8.8.1 legacy-jira-priority:Minor module:facet type:enhancement
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

While working on an unrelated getBulkPath API [PR](https://github.com/apache/lucene/pull/179), @mikemccand and I came across a nice optimization that could be made to the taxonomy cache.

The taxonomy cache today caches frequently used ordinals and their corresponding FacetLabels. It uses the existing LRUHashMap (backed by a LinkedList) class for its implementation.

This implementation performs sub optimally when it has a large number of threads accessing it, and consumes a large amount of RAM.

@mikemccand suggested the idea of a two array backed HPPC int->FacetLabel cache. The basic idea behind the cache being:
1. We use two hashmaps primary and secondary.
1. In case of a cache miss in the primary and a cache hit in the secondary, we add the key to the primary map as well.
1. In case of a cache miss in both the maps, we add it to the primary map.
1. When we reach (make this check each time we insert?) a large number of elements in say the primary cache, (say larger than the existing DEFAULT_CACHE_VALUE=4000), we dump the secondary map and copy all the values of the primary map into it.

The idea was originally explained in [this](https://github.com/apache/lucene/pull/179#discussion_r692907559) comment.

---
Migrated from [LUCENE-10068](https://issues.apache.org/jira/browse/LUCENE-10068) by Gautam Worah (@gautamworah96), updated Sep 23 2021
Attachments: [disable_taxo_category_cache_benchmark](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-10068/disable_taxo_category_cache_benchmark)

Contributor guide

Open the contributing guide

Research direction

Start by locating the taxonomy cache, its LRUHashMap implementation, and DEFAULT_CACHE_VALUE; then read the linked PR discussion for the intended design. Use the attached disable_taxo_category_cache_benchmark as the available performance reference. Done means the cache follows the proposed two-map behavior while addressing the stated concurrency and RAM concerns.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
performance, search
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.