dotnet / dotnet/spark

[FEATURE REQUEST]: Increase number of UDF parameters

Open
#1,005 2 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
C#
Stars
2.1k
Forks
332
Avg merge
1d 20h
Merged PRs (30d)
9

Description

**Is your feature request related to a problem? Please describe.**
UDFs currently have a limit of 10 input parameters. They are wrapped by C# Func<> which allow up to 16 input parameters.

**Describe the solution you'd like**
Increase the number of input parameters in UDFs to have parity with Func.

**Describe alternatives you've considered**
The workaround is to collapse parameter values for calling the UDF and rehydrate them inside the underlying method. This works, but adds overhead to the Notebook.

![image](https://user-images.githubusercontent.com/1847996/145071965-c1dd54a3-7fe0-4485-9d3f-51c3f73af6bf.png)
![image](https://user-images.githubusercontent.com/1847996/145072061-fa0cdfd9-7c08-46bd-8eed-4558a4d67233.png)
![image](https://user-images.githubusercontent.com/1847996/145072492-7d5eaa30-b37d-47d5-878a-99a3801ab355.png)
![image](https://user-images.githubusercontent.com/1847996/145072672-244d011c-59cb-4149-a517-d0b0edb7a454.png)

**Additional context**
Code: [UDF source file](https://github.com/dotnet/spark/blob/c79cf8fad49e4760e6f6b4c8fe31a59f0fa306c4/src/csharp/Microsoft.Spark/Sql/DataFrameUdfWrapper.cs)
Documentation: [.NET Func with 16 inputs](https://docs.microsoft.com/en-us/dotnet/api/system.func-17?view=net-6.0)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.