dotnet / dotnet/spark

[FEATURE REQUEST]: udf support for dynamic assemblies

Open
#957 4 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
C#
Stars
2.1k
Forks
332
Avg merge
1d 20h
Merged PRs (30d)
9

Description

In a recent talk [Michael Rys](https://channel9.msdn.com/Shows/On-NET/Whats-New-in-NET-for-Apache-Spark-v111) showed udfs being used in a notebook setting with Azure Synapse.

I was under the impression udfs worked in compiled assemblies only.

Not sure the how the notebooks are coded but when I try to define a udf in F# interactive, I get the following error:
```
System.NotSupportedException: The invoked member is not supported in a dynamic assembly.
at System.Reflection.Emit.InternalAssemblyBuilder.get_Location()
at Microsoft.Spark.Utils.UdfSerDe.SerializeType(Type type)
```

It would be nice if the creation of Spark udfs in most interactive environments was supported. Let us know if there any plans for this.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.