[BUG]: Better logging when UDF type can't be found
- Dominant language
- C#
- Stars
- 2.1k
- Forks
- 332
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 9
Description
**Describe the bug**
When the SparkWorker.exe and the host runner .exe process load 2 different versions of an assembly containing a UDF, you get an error similar to:
```
ProcessStream() failed with exception: System.ArgumentNullException: Value cannot be null. (Parameter 'type')
at System.Activator.CreateInstance(Type type, BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[] activationAttributes)
at System.Activator.CreateInstance(Type type, BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture)
at Microsoft.Spark.Utils.CommandSerDe.CreateUdfWrapperDelegate[T](Type type, Object[] parameters) in /_/src/csharp/Microsoft.Spark/Utils/CommandSerDe.cs:line 291
at Microsoft.Spark.Utils.CommandSerDe.Deserialize[T](Stream stream, SerializedMode& serializerMode, SerializedMode& deserializerMode, String& runMode) in /_/src/csharp/Microsoft.Spark/Utils/CommandSerDe.cs:line 243
at Microsoft.Spark.Worker.Processor.CommandProcessor.ReadSqlCommands(PythonEvalType evalType, Stream stream) in D:\a\1\s\src\csharp\Microsoft.Spark.Worker\Processor\CommandProcessor.cs:line 160
at Microsoft.Spark.Worker.Processor.CommandProcessor.ReadSqlCommands(PythonEvalType evalType, Stream stream, Version version) in D:\a\1\s\src\csharp\Microsoft.Spark.Worker\Processor\CommandProcessor.cs:line 117
at Microsoft.Spark.Worker.Processor.CommandProcessor.Process(Stream stream) in D:\a\1\s\src\csharp\Microsoft.Spark.Worker\Processor\CommandProcessor.cs:line 62
at Microsoft.Spark.Worker.Processor.PayloadProcessor.Process(Stream stream) in D:\a\1\s\src\csharp\Microsoft.Spark.Worker\Processor\PayloadProcessor.cs:line 74
at Microsoft.Spark.Worker.TaskRunner.ProcessStream(Stream inputStream, Stream outputStream, Version version, Boolean& readComplete) in D:\a\1\s\src\csharp\Microsoft.Spark.Worker\TaskRunner.cs:line 143
```
We should give a better error indicating what we were trying to do when we got an `ArgumentNullException` and what it means.
**To Reproduce**
Steps to reproduce the behavior:
1. Have the SparkWorker.exe and the host runner .exe load 2 different versions of an assembly containing a UDF. Usually the UDF type/method name has changed if you added more anonymous methods to a new version of an assembly.
2. Try to invoke the UDF
3. See error above.
**Expected behavior**
You should get an actionable error indicating what went wrong, and how you can fix it.
Contributor guide
Assessment
This issue has not been assessed yet.