apache / apache/iotdb

the memory in the old generation isn't GC in time, with code sessionPool.showPathsTemplateSetOn

Open
#14,407 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
6.4k
Forks
1.2k
Avg merge
1d 23h
Merged PRs (30d)
115

Description

When run the code below, the memory in the old generation isn't GC in time, which eventually leads to the data node crashing.

Version
version: 1.3.2

### log :
A fatal error has been detected by the Java Runtime Environment:
Internal Error (debug.cpp:362), pid=32562, tid=88579
fatal error: OutOfMemory encountered: Java heap space

JRE version: OpenJDK Runtime Environment Corretto-17.0.11.9.1 (17.0.11+9) (build 17.0.11+9-LTS)
Java VM: OpenJDK 64-Bit Server VM Corretto-17.0.11.9.1 (17.0.11+9-LTS, mixed mode, sharing, tiered, compressed oops, compressed class ptrs, g1 gc, bsd-aarch64)

--------------- S U M M A R Y ------------

Command Line: --add-opens=java.base/java.util.concurrent=ALL-UNNAMED --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.net=ALL-UNNAMED -Dlogback.configurationFile=./sbin/../conf/logback-datanode.xml -DIOTDB_HOME=./sbin/.. -DIOTDB_DATA_HOME=./sbin/.. -DTSFILE_HOME=./sbin/.. -DIOTDB_CONF=./sbin/../conf -DTSFILE_CONF=./sbin/../conf -Dname=iotdb\.IoTDB -DIOTDB_LOG_DIR=./sbin/../logs -DOFF_HEAP_MEMORY=1232M -Diotdb.jmx.local=true -Xms4912M -Xmx4912M -XX:MaxDirectMemorySize=1232M -Djdk.nio.maxCachedBufferSize=1291845 -XX:+CrashOnOutOfMemoryError -XX:+UseAdaptiveSizePolicy -Xss512k -XX:SafepointTimeoutDelay=1000 -XX:+SafepointTimeout org.apache.iotdb.db.service.DataNode -s

Host: "Mac14,7" arm64, 8 cores, 16G, Darwin 23.4.0, macOS 14.4 (23E214)
Time: Thu Dec 12 22:29:33 2024 CST elapsed time: 33.258095 seconds (0d 0h 0m 33s)

### code:
```
public class SessionPoolDemo {

private static SessionPool sessionPool;
private static void constructCustomSessionPool() {
sessionPool = new SessionPool.Builder()
.host("127.0.0.1")
.port(6667)
.user("root")
.password("root")
.build();
}

public static void main(String[] args) throws Exception {
constructCustomSessionPool();

for (int i = 0; i < 1000; i++) { // or 5000
List locationTemplate = sessionPool.showPathsTemplateSetOn("locationTemplate");
System.out.println(locationTemplate.size());
}
}
}
```

### GC:
gc

Contributor guide

Open the contributing guide

Research direction

Start with the SessionPool.showPathsTemplateSetOn call used in the SessionPoolDemo and reproduce the issue on IoTDB 1.3.2 using the supplied JVM settings and GC evidence. Trace allocations and retention across repeated calls; done means the loop no longer causes old-generation growth leading to a Java heap-space failure.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.