RAM issue while loading several PCL in threads
- Dominant language
- C++
- Stars
- 14k
- Forks
- 2.6k
- Avg merge
- 5d 18h
- Merged PRs (30d)
- 6
Description
### Checklist
- [x] I have searched for [similar issues](https://github.com/isl-org/Open3D/issues).
- [x] For Python issues, I have tested with the [latest development wheel](https://www.open3d.org/docs/latest/getting_started.html#development-version-pip).
- [x] I have checked the [release documentation](https://www.open3d.org/docs/release/) and the [latest documentation](https://www.open3d.org/docs/latest/) (for `main` branch).
### Describe the issue
I have a system that captures images every N seconds and updates both the Point Cloud (PCL) and Mesh objects. It’s a multi-threaded system. A new PCL is built from each image, and two mesh objects are merged in a daemon thread.
I’m encountering an issue with RAM not being properly freed after several iterations of this process.
To investigate further, I created a minimal Python script that simply loads a PCL object from a file while monitoring RAM usage using the `psutil` package. I was able to reproduce the problem with this script.
I also came across several related questions on GitHub discussing similar issues, including potential problems with using threads while loading DLL packages (e.g., [ref1](https://stackoverflow.com/questions/8076752/memory-leak-when-using-shared-library-with-thread-local-storage-via-ctypes-in-a) ,[ref2](https://sourceware.org/legacy-ml/libc-help/2011-04/msg00000.html)).
Questions:
Can you please confirm whether this is a known issue?
Do you have any suggested workarounds?
### Steps to reproduce the bug
```python
import psutil
import gc
import threading
import open3d as o3d
def monitor_ram_usage(interval=5.0, duration=100):
process = psutil.Process(os.getpid())
start_time = time.time()
while True:
# Get memory info for the process
mem_info = process.memory_info()
# Convert to MB for better readability
rss_mb = mem_info.rss / (1024 * 1024)
vms_mb = mem_info.vms / (1024 * 1024)
print(f"RAM Usage - RSS: {rss_mb:.2f} MB, VMS: {vms_mb:.2f} MB")
last_ram_val = rss_mb
# Check if monitoring duration has been reached
if duration and (time.time() - start_time >= duration):
print("Monitoring complete.")
break
time.sleep(interval)
pcls = []
lock = threading.Lock()
def load_pcl():
ply_point_cloud = o3d.data.PLYPointCloud()
with lock:
pcls.append(o3d.io.read_point_cloud(ply_point_cloud.path))
monitor_thread = threading.Thread(
target=monitor_ram_usage,
kwargs={"interval": 1, "duration": 501},
daemon=True # Thread will terminate when main program exits
)
monitor_thread.start()
for i in range(500):
print("loading_another_pcl")
threading.Thread(target=load_pcl, daemon=True).start()
time.sleep(1)
with lock:
print(f"number of pcl loaded {len(pcls)}")
if i % 50 == 1:
with lock:
del pcls
pcls = []
gc.collect()
```
### Error message
_No response_
### Expected behavior
I start with a RAM usage of around **325 MB**. During the first 50 iterations, each PCL object adds approximately **14 MB**. After 50 iterations, the memory usage reaches about **1000 MB**. Even after explicitly triggering garbage collection, the RAM usage only drops to **640 MB** - **not back to the original 325 MB** - indicating a potential memory leak or lingering memory usage.
### Open3D, Python and System information
```markdown
- Operating system: Ubuntu 20.04
- Python version: Python 3.11
- Open3D version: 0.19.0
- System architecture: x86
- Is this a remote workstation?: no
- How did you install Open3D?: pip
```
### Additional information
_No response_
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.