[Feature] Implement TableOperations-specific FileIO
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 399
- Forks
- 80
- Avg merge
- 3d 23h
- Merged PRs (30d)
- 15
Description
Willingness to contribute
{"label"=>"Yes. I can contribute a fix for this bug independently.", "value"=>"independent"}
Feature Request Proposal
In order to implement FileIOs that are TableOperations-specific, creating a FileIO instance on a per TableOperations-basis is required.
This is similar to how other Iceberg-compliant Catalogs are implemented: https://github.com/apache/iceberg/pull/10893 .
Motivation
What is the use case for this feature?
This will be required to make OpenHouse data-access-tokens work.
Different tables will require different data-access-tokens to be used by its FileIO implementation.
What component(s) does this feature affect?
-
Table Service: This is the RESTful catalog service that stores table metadata.:services:tables -
Jobs Service: This is the job orchestrator that submits data services for table maintenance.:services:jobs -
Data Services: This is the jobs that performs table maintenance.apps:spark -
Iceberg internal catalog: This is the internal Iceberg catalog for OpenHouse Catalog Service.:iceberg:openhouse -
Spark Client Integration: This is the Apache Spark integration for OpenHouse catalog.:integration:spark -
Documentation: This is the documentation for OpenHouse.docs -
Local Docker: This is the local Docker environment for OpenHouse.infra/recipes/docker-compose -
Other: Please specify the component.
Details
No response
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in the :integration:spark component by locating how FileIO instances are currently created for TableOperations. Compare the intended approach with the referenced Apache Iceberg pull request and trace how table-specific data-access-tokens would be supplied. Done means each table can use its required token through its FileIO implementation, with integration coverage confirming the behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, spark
- Domain
- data
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100