Rely on beam partitioning
Open
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 197
- Forks
- 56
- PR merge metrics
- No merged PRs in 30d
Description
We have implemented a custom JDBC parallel read API because at the time, beam did not support that.
Now beam has readWitPartition supporting Long and DateTime.
We should probably considering adopting this instead
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating DBeam's custom JDBC parallel read API and compare its behavior with Apache Beam's JdbcIO readWithPartitions API linked in the issue. Check the Scio issue for related context. Done means determining whether the custom implementation can be replaced by Beam's API while preserving the existing SQL-to-Avro export behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- databases, distributed-systems
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100