dotnet / dotnet/spark

[FEATURE REQUEST]: Support for Spark 3.3.0 using autogen language bindings similar to SynapseML

Open
#1,100 5 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
C#
Stars
2.1k
Forks
332
Avg merge
1d 20h
Merged PRs (30d)
9

Description

Team:

Spark 3.3.0 has already been released. Is there any timeline for a new version of this library with full compatibility with Spark (including SparkML).

I see that SynapseML has a new autogen technology - ["Automating the .Net for Apache Spark Integration" section at the end of this article](https://devblogs.microsoft.com/dotnet/announcing-synapseml-for-dotnet/) - that enables generating .Net (and Python & R too) bindings with much less effort than manual approach. However, I am not sure if that same technology can be used for generating Spark Core bindings as well (would be awesome if it can!).

Are there any plans to use that technology so that there would be a faster release cadence that can match with Spark releases?

Thanks in advance.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.