[ML] Read one object at a time for incremental training state
Open
>enhancement
- Dominant language
- C++
- Stars
- 157
- Forks
- 67
- Avg merge
- 12h 48m
- Merged PRs (30d)
- 16
Description
Currently, we read one big document for the data summarisation and one big document for the inference model, but there is nothing to stop us reading individual state documents one at a time using the `doc.ParseStream` API. This would significantly cutdown the memory when of restoring state for incremental train since we only inflate the state on-the-fly when reading the dechunked stream. Relates to #1849.
Contributor guide
Assessment
This issue has not been assessed yet.