apache / apache/arrow-java

[Java] Arrow-to-JDBC

Open
#363 19 comments 0 reactions 1 assignee Claimed by @wgtmac View on GitHub
Type: enhancement
Dominant language
Java
Stars
94
Forks
152
Avg merge
3d 16h
Merged PRs (30d)
11

Description

ARROW-1780 reads a query from a JDBC data source and converts the ResultSet to an Arrow VectorSchemaRoot.  However, there is no built-in adapter for writing an Arrow VectorSchemaRoot back to the database.

ARROW-3966 adds JDBC field metadata:
- The Catalog Name
- The Table Name
- The Field Name
- The Field Type

We can use this information to ask for the field information from the database via the [DatabaseMetaData](https://docs.oracle.com/javase/7/docs/api/java/sql/DatabaseMetaData.html) object.  We can then create INSERT or UPDATE statements based on the [list of primary keys](https://docs.oracle.com/javase/7/docs/api/java/sql/DatabaseMetaData.html#getPrimaryKeys(java.lang.String,%20java.lang.String,%20java.lang.String)) in the table:
- If the value in the VectorSchemaRoot corresponding to the primary key is NULL, insert that record into the database.
- If the value in the VectorSchemaRoot corresponding to the primary key is not NULL, update the existing record in the database.

We can also perform the same data conversion in reverse based on the field types queried from the database.

**Reporter**: [Michael Pigott](https://issues.apache.org/jira/browse/ARROW-4144) / @mikepigott

**Note**: *This issue was originally created as [ARROW-4144](https://issues.apache.org/jira/browse/ARROW-4144). Please see the [migration documentation](https://github.com/apache/arrow/issues/14542) for further details.*

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.