[Bug]: JdbcDriver causes metaspace leak
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 196
Description
### What happened?
As of now ,`org.apache.beam.sdk.extensions.sql.impl.JdbcDriver` doesn't de-register itself from `java.sql.DriverManager`. And this can causes metaspace leak, because it holds a ChildFirstClassLoader on a flink runner. If we submit jobs many times to a same flink cluster, corresponding cluster OOM eventually.
### Issue Priority
Priority: 2 (default / most bugs should be filed as P2)
### Issue Components
- [ ] Component: Python SDK
- [X] Component: Java SDK
- [ ] Component: Go SDK
- [ ] Component: Typescript SDK
- [ ] Component: IO connector
- [ ] Component: Beam YAML
- [ ] Component: Beam examples
- [ ] Component: Beam playground
- [ ] Component: Beam katas
- [ ] Component: Website
- [ ] Component: Spark Runner
- [ ] Component: Flink Runner
- [ ] Component: Samza Runner
- [ ] Component: Twister2 Runner
- [ ] Component: Hazelcast Jet Runner
- [ ] Component: Google Cloud Dataflow Runner
Contributor guide
Research direction
Start at org.apache.beam.sdk.extensions.sql.impl.JdbcDriver and inspect how it registers with java.sql.DriverManager. Verify the driver is released after use so repeated jobs on the same Flink cluster do not retain ChildFirstClassLoader instances and eventually cause a metaspace leak.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 62/100