autogluon / autogluon/autogluon

Add batch_size option to `predict` and `predict_proba`

Open
#3,415 1 comment 4 reactions 0 assignees View on GitHub
enhancement module: tabular priority: 1
Dominant language
Python
Stars
10.7k
Forks
1.2k
Avg merge
21h 29m
Merged PRs (30d)
57

Description

Add batch_size option to `predict` and `predict_proba` to reduce peak memory usage without changing the end result of the call.

This may result in slower inference speed, and this mainly matters for very large scale batch inference that might run into memory issues during the data preprocessing stage.

User Quote:

> the data fit in the memory just fine, but it was just for prediction it ran out of memory. i tried writing a for loop to chunk predict and it worked. But I was wondering if autogluon supports batchsize in prediction? the datasize for prediction is 87M with 30 features, when I break it down to batch size of 500K and use a for loop for scoring it was perfectly fine and ran very fast.

Contributor guide

Open the contributing guide

Research direction

Start by locating the predict and predict_proba entry points and tracing how prediction data is processed. Check how batching would affect preprocessing and inference, then verify that batch_size reduces peak memory without changing results and that the existing prediction behavior remains unchanged.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.