dotnet / dotnet/spark

[FEATURE REQUEST]: Delta Lake & Hyperspace version static analyzer

Open
#618 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
C#
Stars
2.1k
Forks
332
Avg merge
1d 20h
Merged PRs (30d)
9

Description

**Is your feature request related to a problem? Please describe.**
As we add support for newer versions of Delta Lake and Hyperspace, we need to track which APIs are available in which versions of those libraries.

For example, some Delta Lake APIs may be available only in Delta Lake 0.7.0+ (targeting Spark 3.0+). If a .NET for Apache Spark user is running Spark 2.4.x and Delta Lake 0.6.1, then they should be able to see which Delta Lake APIs are actually available to them to use.

**Describe the solution you'd like**
We should add annotations to all APIs in `Microsoft.Spark.Extensions.Delta` and `Microsoft.Spark.Extensions.Hyperspace` detailing the required version of Delta Lake and Hyperspace, respectively. With those annotations present, we can add static analysis to validate correctness of API usage at compile time.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.