[Java][arrow-jdbc] Ability to customize JdbcConsumer construction
- Linguagem predominante
- Java
- Estrelas
- 94
- Forks
- 152
- Merge médio
- 3d 16h
- PRs com merge (30d)
- 11
Descrição
### Describe the enhancement requested
I'm working on a project that heavily uses the arrow-jdbc library to convert results from a number of different JDBC drivers to Arrow. When the integration was initially written a bit over 2 years ago it forked `ArrowVectorIterator` to add a number of customization points in the JDBC->Arrow conversion which were missing at the time. I'm updating the project and happily in the intervening time it looks like a lot of this customization is now possible natively via customization points in `JdbcToArrowConfig` - in particular the ability to override `JdbcFieldInfo` per column and most importantly the ability to override a `Function`.
I'd love to fully remove our forked `ArrowVectorIterator` but there's one last customization point we're depending on that I don't see a way to handle currently: The ability to customize the `JdbcConsumer` instances constructed for each column. We depend on overriding these to paper over idiosyncrasies with various JDBC drivers. As an example on the top of my mind this morning: The Snowflake JDBC driver [does not support](https://docs.snowflake.com/en/user-guide/jdbc-api.html#id14) `getBinaryStream` and so is not compatible with the default `BinaryConsumer` and must be overriden with a consumer that uses `getBytes` instead. We also add a layer of wrapping to all of our `JdbcConsumers` to wrap error messages to aid debugging (by adding info on the specific column an error originated from).
Would it be possible to add an extension point to the `JdbcToArrowConfig` to customize this, similar to `jdbcToArrowTypeConverter`? I wish we didn't need it but in practice we've found enough variability across JDBC drivers to necessitate it.
### Component(s)
Java
Guia de contribuição
Direção de pesquisa
Comece lendo ArrowVectorIterator e JdbcToArrowConfig e, em seguida, rastreie como as instâncias de JdbcConsumer são construídas para cada coluna. Defina um ponto de extensão análogo a jdbcToArrowTypeConverter que permita consumers específicos do driver e wrapping, e verifique se o workaround de getBytes do Snowflake pode ser fornecido por meio dele.
Escrita pelo modelo de indexação a partir do texto da issue.
Avaliação
- Stack de tecnologia
- java
- Domínio
- databases
- Tipo de issue
- Funcionalidade
- Dificuldade
- 5/5
- Tempo estimado
- Mais de uma semana
- Status de atividade
- Estagnada
- Clareza
- Razoavelmente clara
- Facilidade para iniciantes
- 38/100