fread - OOM issue
Open
- Dominant language
- C++
- Stars
- 1.9k
- Forks
- 164
- Avg merge
- 7h 31m
- Merged PRs (30d)
- 1
Description
**Issue:**
+1 for the awesome package.
Encountering Out Of Memory issue when reading a LARGE csv file (~200 million rows) using fread.
Datatable code is being run on an **_aks-cluster_** with 20Gi of RAM and its still tripping
**Code:**
csv file:
**Expected behavior:**
Being able to load the file and perform manipulations.
**Environment:**
datatable version - 1.0.0
python version - 3.8.0
operating system - linux
Any pointers on how to do the aforementioned task in a memory efficient way would be of great help.
Thanks.
Contributor guide
Assessment
This issue has not been assessed yet.