isl-org / isl-org/Open3D

RAM issue while loading several PCL in threads

Open
#7,222 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
C++
Stars
14k
Forks
2.6k
Avg merge
5d 18h
Merged PRs (30d)
6

Description

### Checklist

- [x] I have searched for [similar issues](https://github.com/isl-org/Open3D/issues).
- [x] For Python issues, I have tested with the [latest development wheel](https://www.open3d.org/docs/latest/getting_started.html#development-version-pip).
- [x] I have checked the [release documentation](https://www.open3d.org/docs/release/) and the [latest documentation](https://www.open3d.org/docs/latest/) (for `main` branch).

### Describe the issue

I have a system that captures images every N seconds and updates both the Point Cloud (PCL) and Mesh objects. It’s a multi-threaded system. A new PCL is built from each image, and two mesh objects are merged in a daemon thread.

I’m encountering an issue with RAM not being properly freed after several iterations of this process.

To investigate further, I created a minimal Python script that simply loads a PCL object from a file while monitoring RAM usage using the `psutil` package. I was able to reproduce the problem with this script.

I also came across several related questions on GitHub discussing similar issues, including potential problems with using threads while loading DLL packages (e.g., [ref1](https://stackoverflow.com/questions/8076752/memory-leak-when-using-shared-library-with-thread-local-storage-via-ctypes-in-a) ,[ref2](https://sourceware.org/legacy-ml/libc-help/2011-04/msg00000.html)).

Questions:

Can you please confirm whether this is a known issue?

Do you have any suggested workarounds?

### Steps to reproduce the bug

```python
import psutil
import gc
import threading
import open3d as o3d

def monitor_ram_usage(interval=5.0, duration=100):
process = psutil.Process(os.getpid())
start_time = time.time()

while True:
# Get memory info for the process
mem_info = process.memory_info()

# Convert to MB for better readability
rss_mb = mem_info.rss / (1024 * 1024)
vms_mb = mem_info.vms / (1024 * 1024)

print(f"RAM Usage - RSS: {rss_mb:.2f} MB, VMS: {vms_mb:.2f} MB")
last_ram_val = rss_mb

# Check if monitoring duration has been reached
if duration and (time.time() - start_time >= duration):
print("Monitoring complete.")
break

time.sleep(interval)
pcls = []
lock = threading.Lock()

def load_pcl():
ply_point_cloud = o3d.data.PLYPointCloud()
with lock:
pcls.append(o3d.io.read_point_cloud(ply_point_cloud.path))

monitor_thread = threading.Thread(
target=monitor_ram_usage,
kwargs={"interval": 1, "duration": 501},
daemon=True # Thread will terminate when main program exits
)

monitor_thread.start()
for i in range(500):
print("loading_another_pcl")
threading.Thread(target=load_pcl, daemon=True).start()
time.sleep(1)
with lock:
print(f"number of pcl loaded {len(pcls)}")
if i % 50 == 1:
with lock:
del pcls
pcls = []
gc.collect()
```

### Error message

_No response_

### Expected behavior

I start with a RAM usage of around **325 MB**. During the first 50 iterations, each PCL object adds approximately **14 MB**. After 50 iterations, the memory usage reaches about **1000 MB**. Even after explicitly triggering garbage collection, the RAM usage only drops to **640 MB** - **not back to the original 325 MB** - indicating a potential memory leak or lingering memory usage.

### Open3D, Python and System information

```markdown
- Operating system: Ubuntu 20.04
- Python version: Python 3.11
- Open3D version: 0.19.0
- System architecture: x86
- Is this a remote workstation?: no
- How did you install Open3D?: pip
```

### Additional information

_No response_

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.