apache / apache/polaris

Polaris Spark Catalog can't handle procedures

Open
#2,550 2 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Java
Stars
2.1k
Forks
522
Avg merge
1d 22h
Merged PRs (30d)
137

Description

### Describe the bug

In most cases Polaris SparkCatalog acts as a wrapper around the iceberg catalog with added functionality. However Polaris catalog plugin currently doesn't handle procedures while iceberg implementation does. For v3.5 Iceberg uses it's own ProcedureCatalog, while for v4.0 the one in spark. It should be possible (?) to mimic iceberg behavior even in current 3.5 version by utilizing ProcedureCatalog interface from iceberg.

### To Reproduce

Call any of the procedures from [iceberg](https://iceberg.apache.org/docs/latest/spark-procedures/).

### Actual Behavior

Polaris SparkCatalog throws an error.

### Expected Behavior

Polaris SparkCatalog should delegate procedure calls to iceberg

### Additional context

_No response_

### System information

_No response_

Contributor guide

Open the contributing guide

Research direction

Start at the Polaris SparkCatalog entry point and compare its procedure handling with the Iceberg Spark procedures described in the linked documentation. Check the version-specific ProcedureCatalog behavior for Spark 3.5, then verify that the listed Iceberg procedures are delegated without the current error.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.