[FEATURE REQUEST]: Add Application .Zip file contents to Assembly Search Path
- Dominant language
- C#
- Stars
- 2.1k
- Forks
- 332
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 9
Description
**Is your feature request related to a problem? Please describe.**
This is in relation to one of the new features added in v0.4.0:
> - The ability to use `--archives` option to deploy the worker binaries and assemblies that UDFs depend on (#187)
The `--archives` option only works for YARM setups. Databricks, for example, does not use YARM so the `--archives` parameter has no effect.
**Describe the solution you'd like**
For the application .zip file to be transferred to all workers, extracted and the contents to be in the assembly search path.
**Describe alternatives you've considered**
A partial workaround is adding the applications .zip file name to the `--files` parameter meets the first requirement of transferring it to all workers. However, it is not automatically extracted or added to the assembly search path.
Databricks Spark-Submit example:
```
["--files","DotNetSparkTest.zip","--class","org.apache.spark.deploy.dotnet.DotnetRunner","/dbfs/spark-dotnet/microsoft-spark-2.4.x-0.4.0.jar","/dbfs/apps/DotNetSparkTest.zip","DotNetSparkTest"]
```
Contributor guide
Assessment
This issue has not been assessed yet.