Azure / Azure/MachineLearningNotebooks

DataBricksStep never ends

Open
#1,603 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
4.4k
Forks
2.6k
PR merge metrics
No merged PRs in 30d

Description

Hi!
I've an AzureML pipeline with several DataBricksSteps. The former work perfectly but the latter never stops. It's very strange because I know that the script has finished running (I put a log at the end and I see it in my log file).
I tried with different versions of Azure packages (azureml-sdk, azureml-opendatasets ...) but it doesn't change anything. For information, the configuration of the DB cluster used for all steps is:

- spark_version = 7.3.x-scala2.12
- node_type = Standard_DS3_v2

Any idea what's wrong? Or is there a way to force the step to stop without canceling it?

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.