apache / apache/incubator-xtable

Introduce Injection to add config values from PerTableConfig to TargetClients

Open
#306 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
1.2k
Forks
212
Avg merge
4d 9h
Merged PRs (30d)
16

Description

Since #304 split the TargetClient creation and initialization into separate steps with a create and init() method, we not have an opportunity to remove the direct dependency on PerTableConfig from the api module and the init(). Even though #304 added a new interface to the api module for PerTableConfig being added to the init() signature, it has proven difficult to separate table format specifics from the interface. A separate issue has been filed in #297 to help resolve that and improve extensibility.

This issue represents a change that will allow TargetClients to simply implement setters for each of the getters in PerTableConfig that it requires and they will magically be set prior to calling the init(). The init() will then be able to rely on them having already been set and use them as needed to complete initialization.

It will remove the need for the interface layer to be aware of target format config specifics and the need to pass PerTableConfig to the init().

Open Questions:
1. Should we move each of the targetClients that uses the Hadoop Configuration class to have a setter and remove that from the init() arguments as well?
2. Should we revisit the public factory method to createTargetClientForName() and only rely on PerTableConfig and the builtin injection or should we keep the flexibility to manually do the setting of those params?
3. Should we keep the PerTableConfig interface that was added to help determine all the getters for which we will attempt to use setters on each targetClient or should we remove it and add more logic in order to filter out the setters from the getters scraped from the PerTableConifgImpl?
4. Should we remove the HudiSourceConfig and CatalogConfig interfaces that were previously added so that the implementations wouldn't be pulled into the api module by the PerTableConfig interface?

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reading the API module's TargetClient creation and init() flow, the PerTableConfig interface, and the target clients that consume its getters. Resolve the four open design questions before changing the injection approach. Done means target-specific configuration is injected through setters before init(), without passing PerTableConfig to init().

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend-api-design
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.