apache / apache/gravitino

[Improvement] Refactor the table implementation in spark-connector

Open
#2,845 0 comments 0 reactions 0 assignees View on GitHub
improvement
Dominant language
Java
Stars
3.2k
Forks
935
Avg merge
1d 15h
Merged PRs (30d)
315

Description

### What would you like to be improved?

Refactor the table implementation in spark-connector.

The purpose of inheritance `SparkTable` is in order to go through the `isIcebergTable` check in `IcebergSparkSqlExtensionsParser`.

For details, please refer to: https://github.com/apache/iceberg/blob/main/spark/v3.4/spark-extensions/src/main/scala/org/apache/spark/sql/catalyst/parser/extensions/IcebergSparkSqlExtensionsParser.scala#L127-L186

Without refactoring, row-level operations cannot be performed.

### How should we improve?

_No response_

Contributor guide

Open the contributing guide

Research direction

Start by reading IcebergSparkSqlExtensionsParser.scala at the linked lines, then locate the SparkTable implementation in spark-connector and determine how the isIcebergTable check relates to it. The issue provides no named tests or target files; done means the refactored table implementation permits row-level operations.

Written by the indexing model from the issue text.

Assessment

Tech stack
scala, spark
Domain
data-engineering
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.