apache / apache/incubator-xtable
Ability to create TargetTable's metadata in a different location from the SourceTable
- Dominant language
- Java
- Stars
- 1.2k
- Forks
- 212
- Avg merge
- 4d 9h
- Merged PRs (30d)
- 16
Description
OneTable's translation flow currently assumes that the source table directory can be modified. However, this may not always be the case. For example, the source table may contain critical data that should not be corrupted. Therefore, the user may not have the permission to write to the source table's location. This would cause OneTable to fail. A better alternative would be to allow the target table's metadata to be created in a different location than the source table.
Additionally, if the metadata and data directories are on different paths, some table formats might not work. The `TargetClient` should provide a mechanism to perform this check and return an error if the target format cannot support it.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reading the translation flow and the TargetClient interface mentioned in the issue. Trace where target metadata is currently written relative to the source table, then define how an alternate metadata location and unsupported format validation should behave; done means read-only source locations work and incompatible metadata/data paths return an error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering, databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100