autogluon / autogluon/autogluon
Add batch_size option to `predict` and `predict_proba`
- Dominant language
- Python
- Stars
- 10.7k
- Forks
- 1.2k
- Avg merge
- 21h 29m
- Merged PRs (30d)
- 57
Description
Add batch_size option to `predict` and `predict_proba` to reduce peak memory usage without changing the end result of the call.
This may result in slower inference speed, and this mainly matters for very large scale batch inference that might run into memory issues during the data preprocessing stage.
User Quote:
> the data fit in the memory just fine, but it was just for prediction it ran out of memory. i tried writing a for loop to chunk predict and it worked. But I was wondering if autogluon supports batchsize in prediction? the datasize for prediction is 87M with 30 features, when I break it down to batch size of 500K and use a for loop for scoring it was perfectly fine and ran very fast.
Contributor guide
Research direction
Start by locating the predict and predict_proba entry points and tracing how prediction data is processed. Check how batching would affect preprocessing and inference, then verify that batch_size reduces peak memory without changing results and that the existing prediction behavior remains unchanged.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100