microsoft / microsoft/SynapseML

Need to perform incremental training

Open
#2,340 0 comments 0 reactions 0 assignees View on GitHub
triage
Dominant language
Scala
Stars
5.2k
Forks
868
Avg merge
22h 9m
Merged PRs (30d)
45

Description

Taking the regression model as an example, it is recommended to add a parameter "task" when instantiating LightGBMRegressor. I found that I couldn't continue to perform incremental training on the saved models in the form of a string or a file during my use. The LightGBM algorithm in Microsoft includes this part, and I hope to see it gradually supplemented.

task [🔗︎](https://lightgbm.readthedocs.io/en/latest/Parameters.html#task), default = train, type = enum, options: train, predict, convert_model, refit, aliases: task_type

train, for training, aliases: training

predict, for prediction, aliases: prediction, test

convert_model, for converting model file into if-else format, see more information in [Convert Parameters](https://lightgbm.readthedocs.io/en/latest/Parameters.html#convert-parameters)

refit, for refitting existing models with new data, aliases: refit_tree

save_binary, load train (and validation) data then save dataset to binary file. Typical usage: save_binary first, then run multiple train tasks in parallel using the saved binary file

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the SynapseML LightGBM regression integration and the linked LightGBM parameters documentation, especially the refit task. Investigate how saved models are represented and loaded when using LightGBMRegressor. Done means incremental or refit training is supported for saved models supplied as strings or files, with behavior covered by appropriate tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
scala, spark
Domain
machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.