apache / apache/gravitino

[Improvement] Read the latest scheme for an external Lance table via the location when loading it

Open
#9,706 5 comments 0 reactions 0 assignees View on GitHub
improvement
Dominant language
Java
Stars
3.2k
Forks
935
Avg merge
1d 17h
Merged PRs (30d)
339

Description

### What would you like to be improved?

Currently, the Lance table in Lakehouse generic support properties `external`, that is
- `true` means the location is not fully managed by the Gravitino server, and Users can access the location directly without using Gravitino API
- `false` means that the location is only managed and accessible by the Gravitino server.

For a lance table with `external = true`, **we need to access the location to get the latest metadata when trying to load the table.**

### How should we improve?

_No response_

Contributor guide

Open the contributing guide

Research direction

Start by tracing the Lance table loading path and how the external generic support property is read. Follow how table metadata is obtained from the configured location, then verify that loading an external Lance table observes the latest metadata rather than only server-managed state.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
data-engineering, databases
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.