[Feature] Widen foreign key dimension table
- Dominant language
- Java
- Stars
- 3.4k
- Forks
- 1.4k
- Avg merge
- 1d 11h
- Merged PRs (30d)
- 396
Description
### Search before asking
- [X] I searched in the [issues](https://github.com/apache/incubator-paimon/issues) and found nothing similar.
### Motivation
Widen a foreign key dimension table by flink stream join job is too expensive. We can handle this by join two tables in paimon storage.
For example:
Table 1: id name (primary key is id)
Table 2: out_id my_id price (primary key is my_id, out_id is a foreign key of Table 1 id)
We want Table 3: out_id my_id price name
Through flink join job, it's expensive for resource(join waste memory).
### Solution
Not clear now, maybe we can create paimon view or something like this.
### Anything else?
_No response_
### Are you willing to submit a PR?
- [ ] I'm willing to submit a PR!
Contributor guide
No contributing guide indexed for this repository
Research direction
The issue names no files, tests, or entry points, and its proposed solution is explicitly undecided. Start by clarifying whether a Paimon view or another storage-level join is intended, then define the supported foreign-key behavior and how completion will be verified.
Written by the indexing model from the issue text.
Assessment
- Domain
- data-engineering, databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100