apache / apache/accumulo

Remove reliance on java gc cleaners in NativeMap code

Open
#4,911 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Java
Stars
1.2k
Forks
487
Avg merge
4d 7h
Merged PRs (30d)
15

Description

**Describe the bug**

The NativeMap code creates iterators over the native map that have a java component and a native component. The native component has a small amount of native memory allocated that the java code references and frees. AFAICT freeing these native iterators is handled in the code by Java GC cleaners, this needs to be double checked. These iterators can be created really frequently creating a lot of small work for the Java GC cleaners. This does not seem like a good practice and it is unclear what the impact on Java GC is by making it do all of this work. It is likely that the impact can change depending on java gc settings, so it is not good for predictability.

**Expected behavior**

The Native map code makes a best effort attempt to always free the native iterator memory for normal operations and only relies on the java gc cleaners for exceptional edge cases.

The following native map code is involved in this.

NativeMap.NMIterator
NativeMap.ConcurrentIterator
NativeMap.createNMI
NativeMap.deleteNMI

These native map iterators are used by scans and minor compactions which have well defined life cycles. Hopefully can refactor the code to free native memory when the iterator is no longer used.

Contributor guide

Open the contributing guide

Research direction

Start by tracing NativeMap.NMIterator, NativeMap.ConcurrentIterator, NativeMap.createNMI, and NativeMap.deleteNMI through scans and minor compactions. Verify how iterator lifecycles currently interact with Java GC cleaners, then define cleanup at normal lifecycle boundaries while retaining cleaners for exceptional cases. Done means normal operations make a best-effort release of native iterator memory.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases, performance
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.