Azure / Azure/MachineLearningNotebooks

Can't use OutputFileDatasetConfig as AutoML pipeline input

Offen
#1,607 5 Kommentare 2 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
Vorherrschende Sprache
Jupyter Notebook
Sterne
4.4k
Forks
2.6k
PR-Merge-Kennzahlen
Keine gemergten PRs in 30 T.

Beschreibung

We have been trying to use the `OutputFileDatasetConfig` of a `ParallelRunStep` as input to an `AutoMLStep` and found that we couldn't; it appears not to work as intended. The input to AutoML needs to be a registered Dataset, but calling register_on_complete on the `OutputFileDatasetConfig` isn't enough to be able to reuse it.

Like others logging issues here, we've been struggling to get the AutoML part of the SDK to work as the documentation suggests it should, and are running into limitations around column type inference that aren't apparent when you drive it via the web studio.

* We spent a lot of time creating custom FeaturizationConfig to detect column types while trying to resolve the issue of being unable to directly reuse a pipeline created tabular dataset.
* The `label_column_name` parameter doesn't accept integers as column indexes for datasets without header rows as the documentation says it should.
* `read_delimited_files()` method on `OutputFileDatasetConfig` has different behaviour to `from_delimited_files()` on `TabularDatasetFactory` and won't produce the right inferred column types for a large number of columns.

In outline what we are doing is this:

```
# This step produces a CSV with a label and 1600 numeric columns
features_output = OutputFileDatasetConfig(name="feature_extraction").read_delimited_files().register_on_complete('dataset-name')
batch_feature_extract_step = ParallelRunStep(
name=parallel_step_name,
inputs=[previous_output.as_input()],
output=features_output,
parallel_run_config=parallel_run_config,
allow_reuse=False
)

automl_settings = {​​​​​​​
"experiment_timeout_minutes": 40,
"max_concurrent_iterations": 4,
"primary_metric" : 'accuracy'
}​​​​​​​

featurization_config = FeaturizationConfig()
for i in range(2, 1602):
featurization_config.add_column_purpose(f'Column{i}', 'Numeric')

# This step attempts to use the previous step's output as input
automl_config = AutoMLConfig(compute_target=compute_target,
task = "classification",
path = "../src",
training_data=features_output.as_input()
label_column_name="Column1", # int doesn't work like docs say
enable_early_stopping= True,
featurization=featurization_config, # turn off leads to error
debug_log = "automl_errors.log",
test_size = 0.1,
validation_size= 0.1,
**automl_settings
)

automl_step = AutoMLStep(
name='automl_classification',
automl_config=automl_config,
outputs=[model_output, metrics_output],
allow_reuse=True)

pipeline = Pipeline(workspace=ws, steps=[parallel_step, automl_step])
```

We're seeing errors like this:

```
raise ValueError("The DatasetConsumptionConfig for {} must be constructed with a ".format(arg) +
ValueError: The DatasetConsumptionConfig for training_data must be constructed with a TabularDataset or OutputTabularDatasetConfig.
```

Even though we are calling `read_delimited_files` on the OutputFileDatasetConfig to convert it to tabular form as the documentation suggests (with sparse detail) we should.

Feeding AutoML the output of a previous pipeline step seems like a reasonable thing to want to do; is there a better suggested workaround than breaking our pipeline in half? We've also seen the behaviours in #1494 and #1605 - it generally feels like the AutoML SDK is undercooked, what are we missing that the web studio internally is doing? Is it possible to provide a more informative notebook for those of us struggling with it, than the one available in this repository?

https://docs.microsoft.com/en-us/python/api/azureml-core/azureml.data.output_dataset_config.outputtabulardatasetconfig?view=azure-ml-py

Beitragsleitfaden

Für dieses Repository ist kein Beitragsleitfaden indexiert

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.