Azure / Azure/MachineLearningNotebooks
DataBricksStep has deprecated init_scripts incorrectly
- 主要语言
- Jupyter Notebook
- 星标
- 4.4k
- 派生
- 2.6k
- PR 合并指标
- 30 天内没有已合并 PR
描述
The argument init_script is no longer working for DataBricksStep(). The docs cite the following reason.
init_scripts | [str]Deprecated. Databricks announced the init script stored in DBFS will stop work after Dec 1, 2023. To mitigate the issue, please 1) use global init scripts in databricks following https://learn.microsoft.com/azure/databricks/init-scripts/global 2) comment out the line of init_scripts in your AzureML databricks step.
However, init_scripts have not been deprecated by DataBricks. DataBricks has deprecated storing init_scripts in dbfs:/ location, and instead suggets to store them in Workspace folders. This is what DataBricks recommends for all clusters and is, in fact, the only possible way for Unity Catalog enabled DataBricks clusters as global init scripts does not work for them.
Please enable init_scripts parameter for the DataBricksStep() as the current way of using global init_scripts does not work for Unity Catalog enabled clusters.
贡献指南
这个仓库没有索引到贡献指南
调研方向
首先定位 DataBricksStep 及其对参数 init_scripts 的处理,然后将当前文档与链接的 Databricks init script 指南进行比较。当 Workspace 文件夹中的 init script 对启用了 Unity Catalog 的集群仍然可用,并且弃用指南准确描述 DBFS 存储而不是禁用 init_scripts 时,工作即告完成。
由索引模型根据 Issue 内容生成。
评估
- 技术栈
- azure, python
- 领域
- cloud, machine-learning
- Issue 类型
- 功能
- 难度
- 3/5
- 预计耗时
- 1-2 天
- 活跃度
- 停滞
- 描述清晰度
- 基本清楚
- 新手友好度
- 38/100