h2oai / h2oai/datatable

fread memory_limit arg could rbind into single temp jay

Open
#2,822 2 comments 0 reactions 0 assignees View on GitHub
fread new feature
Dominant language
C++
Stars
1.9k
Forks
164
Avg merge
7h 31m
Merged PRs (30d)
1

Description

> memory_limit int
...
In the end the returned Frame will be partially composed from data located on disk, and partially from the data in memory. It is advised to either store this data as a Jay file or filter and materialize the frame (if not the performance may be slow).

This FR asks for option for automatic storing data (AFAIU rbind) into single temporary jay file. Could be even the default IMO

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.