apache / apache/incubator-xtable

Ability to create TargetTable's metadata in a different location from the SourceTable

Open
#296 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Java
Stars
1.2k
Forks
212
Avg merge
4d 9h
Merged PRs (30d)
16

Description

OneTable's translation flow currently assumes that the source table directory can be modified. However, this may not always be the case. For example, the source table may contain critical data that should not be corrupted. Therefore, the user may not have the permission to write to the source table's location. This would cause OneTable to fail. A better alternative would be to allow the target table's metadata to be created in a different location than the source table.

Additionally, if the metadata and data directories are on different paths, some table formats might not work. The `TargetClient` should provide a mechanism to perform this check and return an error if the target format cannot support it.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reading the translation flow and the TargetClient interface mentioned in the issue. Trace where target metadata is currently written relative to the source table, then define how an alternate metadata location and unsupported format validation should behave; done means read-only source locations work and incompatible metadata/data paths return an error.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
data-engineering, databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.