Azure / Azure/azureml-examples

pipeline_with_hyperparameter_sweep does not work with function components

Open
#1,697 2 comments 1 reaction 0 assignees View on GitHub
example issue pipeline
Dominant language
Jupyter Notebook
Stars
2k
Forks
1.7k
Avg merge
18h 18m
Merged PRs (30d)
2

Description

## Which example? Describe the issue

example:
pipeline_with_hyperparameter_sweep

description:
Hyperparameter sweep seems to only work for the component defined by script and yml file. If the component is a python function component, the parameters selected for each trial cannot be feed correctly into the command job.

Not sure if the hyperparameter sweep is "by design" only for the script components or it is a bug.

## Additional context

Code that defines the function component instead of script component in example:
```python
import mlflow
from mldesigner import command_component, Input, Output
# conda_env defined with conda.yml file...
@command_component(environment=conda_env)
def train_component_func(
data: Input(type="uri_folder"),
# model_output: Output(type="mlflow_model"),
test_data: Output(type="uri_folder"),
c_value:float=1.0,
kernel:str="rbf",
degree:int=3,
gamma:str="scale",
coef0:float=0,
shrinking:bool=False,
probability:bool=False,
tol:float=1e-3,
cache_size:float=1024,
# class_weight:Dict=None,
verbose:bool=False,
max_iter:int= -1,
decision_function_shape:str="ovr",
break_ties:bool=False,
random_state:int=42
):
# enable auto logging
mlflow.autolog()

# setup parameters
params = {
"C": c_value,
"kernel": kernel,
"degree": degree,
"gamma": gamma,
"coef0": coef0,
"shrinking": shrinking,
"probability": probability,
"tol": tol,
"cache_size": cache_size,
#"class_weight": class_weight,
"verbose": verbose,
"max_iter": max_iter,
"decision_function_shape": decision_function_shape,
"break_ties": break_ties,
"random_state": random_state,
}
print("This is a function internal print of hyperparams")
print(params)

# read in data
df = pd.read_csv(data)

# process data
X_train, X_test, y_train, y_test = process_data(df, random_state)

# train model
model = train_model(params, X_train, X_test, y_train, y_test)
print(model.get_params())
# Output the model and test data
# mlflow.sklearn.save_model(model, model_output)
# X_test.to_csv(Path(test_data) / "X_test.csv", index=False)
# y_test.to_csv(Path(test_data) / "y_test.csv", index=False)
# rest the same as in example

```

Code for pipeline submitting
```python
# the same as in original notebook except:
from train_src.train import train_component_func
@pipeline()
def pipeline_with_hyperparameter_sweep():
"""Tune hyperparameters using func components."""
train_model = train_component_func(
data=Input(
type="uri_file",
path="wasbs://datasets@azuremlexamples.blob.core.windows.net/iris.csv",
),
c_value=Uniform(min_value=0.5, max_value=0.9),
kernel=Choice(["rbf", "linear", "poly"]),
coef0=Uniform(min_value=0.1, max_value=1),
degree=3,
gamma="scale",
shrinking=False,
probability=False,
tol=0.001,
cache_size=1024,
verbose=False,
max_iter=-1,
decision_function_shape="ovr",
break_ties=False,
random_state=42,
)

sweep_step = train_model.sweep(
primary_metric="training_f1_score",
goal="minimize",
sampling_algorithm="random",
compute="cpu_cluster",
)
sweep_step.set_limits(max_total_trials=20, max_concurrent_trials=10, timeout=7200)

pipeline_job = pipeline_with_hyperparameter_sweep()
```
Actual results in Studio:
As can be seen, the print out of model parameters differ from in the trial overview, and it is for all the trials the same (default parameters).
Screenshot 2022-09-29 at 17 05 36

-

Contributor guide

Open the contributing guide

Research direction

Start with the pipeline_with_hyperparameter_sweep example and the train_component_func definition imported from train_src.train. Reproduce the submitted sweep and compare each trial's selected values with the function's printed model parameters. Done means function components receive the trial-specific hyperparameters rather than the same defaults in every trial.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure, python
Domain
cloud, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.