apache / apache/druid

Optimize memory usage by AtomicUpdateGroup

Open
#8,168 1 comment 0 reactions 0 assignees View on GitHub
Contributions Welcome Performance
Dominant language
Java
Stars
14.1k
Forks
3.8k
Avg merge
2d 58m
Merged PRs (30d)
233

Description

There is a comment inside `AtomicUpdateGroup` talking about the importance of memory usage by this class.

The Javadoc comment for the class also says that this object usually contains just one entry.

If so, the easy optimization is to initialize `chunks = new ArrayList<>(1)` - with the capacity of 1.

A more advanced version would be for `OvershadowableManager` to store `Short2ObjectSortedMap` maps, where the values may be either `PartitionChunk` directly (if there is one chunk in a group), or `AtomicUpdateGroup` (if there are 2+ chunks in a group). It would complicate ` OvershadowableManager`'s code but would allow avoiding a level of indirection and creation of a lot of `AtomicUpdateGroup`, `ArrayList`, and `Object[]` objects.

FYI @jihoonson

Contributor guide

Open the contributing guide

Research direction

Search for AtomicUpdateGroup and OvershadowableManager, then read the class comment and the existing memory-usage comment to understand the intended representation. Decide which of the proposed optimizations is in scope; done means the selected memory reduction is implemented without changing update-group behavior and is validated by the relevant tests or measurements.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
performance
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
32/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.