[QST] [libcudf & nsys] Unstable Runtime during Parquet Reading: due to I/O Futex and munmap
- Dominant language
- C++
- Stars
- 9.8k
- Forks
- 1.1k
- Avg merge
- 3d 6m
- Merged PRs (30d)
- 278
Description
I have repeatedly observed unstable runtime when reading Parquet files, where the reading time occasionally exceeds the average. To investigate this issue, I conducted an nsys profiling analysis. The setup is the same as my previous posts as #18278
It's possible that this instability is a profiling-related problem. However, I am unable to confirm this at present.
During the nsys profiling, I frequently encountered two time-consuming issues: I/O futex and Munmap. I am primarily interested in understanding how to eliminate these issues users require a stable performance evaluation without these "noisy" factors. Additionally, I would like to know if you have also encountered these problems. If you have suggestions for a better setup to avoid these time-consuming issues, your input would be greatly appreciated.
# I/O Futex
On my end, I/O futex issues are quite common. For instance, a read operation involving futex from KVIKIO often takes significantly longer than other normal read operations. It appears that this operation is synchronizing with a write call. This typically occurs when reading compressed data, prior to the execution of decoding and decompression kernels.


# Munmap
Munmap issues are less frequent. They occur during the resource deallocation process at the very end when objects are being deconstructed `~memory_mapped_source()`. It is not entirely clear to me whether this unmapping operation is synchronizing with other processes.

---
Finally, if these issues do not impact performance, are merely nsys overheads, or are the result of a misconfiguration in my setup, I will close this inquiry. Thank you.
Contributor guide
Assessment
This issue has not been assessed yet.