dotnet / dotnet/spark

[FEATURE REQUEST]: Spark version 3.1.3 is not supported by current dotnet on spark code. This is preventing Migration to HDI 5.0 which uses spark version 3.1.3

Open
#1,161 2 comments 1 reaction 0 assignees View on GitHub
enhancement
Dominant language
C#
Stars
2.1k
Forks
332
Avg merge
1d 20h
Merged PRs (30d)
9

Description

**Is your feature request related to a problem? Please describe.**

Spark version 3.1.3 is not supported by current dotnet on spark code. This is preventing Migration to HDI 5.0 which uses spark version 3.1.3

**Describe the solution you'd like**
Adding 3.1.3 as supported verion in dotnetrunner class will help resolve this issue.

**Describe alternatives you've considered**
Nil

**Additional context**
Nil

Contributor guide

Open the contributing guide

Research direction

Start by locating the DotnetRunner class and reviewing how existing Spark versions are represented and validated. Add Spark 3.1.3 to that supported-version path, then verify through the repository’s existing validation or test path that the version is accepted.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp
Domain
data-engineering
Issue type
Feature
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.