dotnet / dotnet/spark

Support for NotebookUtils

Open
#1,145 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C#
Stars
2.1k
Forks
332
Avg merge
1d 20h
Merged PRs (30d)
9

Description

Hi

Unfortunately notebookutils is not supported under .net spark
[https://learn.microsoft.com/en-us/azure/synapse-analytics/spark/microsoft-spark-utilities?pivots=programming-language-csharp#notebook-utilities](url)

I would like to use Notebook.Run() inside a task factory as this would allow multiple notebook contexts to run under a single spark application.
[https://learn.microsoft.com/en-us/dotnet/api/system.threading.tasks.taskfactory?view=netcore-3.1#definition](url)
This can be done in PySpark, but .net core 3.1 taskfactory is superior.

When can we expect this support? Is it coming with .net6/7 spark support?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.