GoogleCloudPlatform / GoogleCloudPlatform/professional-services-data-validator

Dataflow Template: Support additional JDBC sources

Open
#848 0 comments 0 reactions 1 assignee Assigned to @ajwelch4 View on GitHub
dataflow template
Dominant language
Python
Stars
524
Forks
171
Avg merge
5d 15h
Merged PRs (30d)
4

Description

Currently, the Dataflow Template only supports BigQuery via the BQ JDBC drivers. We should support all of the sources that DVT has row hash support for. The first step would be to add the additional sources' JDBC drivers as a dependency to the [pom.xml](https://github.com/ajwelch4/DataflowTemplates/blob/b7b7a8530f2ce0ff0042f7027bc1f151ba4f4b85/v2/data-validation/pom.xml). The second step would be to modify the [`getDataSourceConfiguration`](https://github.com/ajwelch4/DataflowTemplates/blob/b7b7a8530f2ce0ff0042f7027bc1f151ba4f4b85/v2/data-validation/src/main/java/com/google/cloud/teleport/v2/templates/DataValidation.java#L285-L304) method to construct the proper `DataSourceConfiguration` instances given the contents of the DVT connection file stored on GCS.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.