apache / apache/gravitino

[Improvement] Trino hive catalog, 'AVRO' format table not support 'tinyint', 'smallint' data type

Open
#5,555 1 comment 0 reactions 0 assignees View on GitHub
improvement
Dominant language
Java
Stars
3.2k
Forks
935
Avg merge
1d 16h
Merged PRs (30d)
298

Description

### What would you like to be improved?

```
create table test_avro_1 (a tinyint,b smallint) WITH (format='AVRO');
CREATE TABLE
trino:default> insert into test_avro_1 values (1,2);
INSERT: 1 row

Query 20241112_110833_00335_trhz3, FINISHED, 1 node
http://localhost:8080/ui/query.html?20241112_110833_00335_trhz3
Splits: 7 total, 7 done (100.00%)
CPU Time: 0.0s total, 0 rows/s, 0B/s, 1% active
Per Node: 0.0 parallelism, 0 rows/s, 0B/s
Parallelism: 0.0
Peak Memory: 695B
0.74 [0 rows, 0B] [0 rows/s, 0B/s]

trino:default> select * from test_avro_1;
a | b
---+---
(0 rows)

Query 20241112_110849_00338_trhz3, FAILED, 1 node
http://localhost:8080/ui/query.html?20241112_110849_00338_trhz3
Splits: 1 total, 1 done (100.00%)
CPU Time: 0.0s total, 200 rows/s, 132KB/s, 35% active
Per Node: 0.0 parallelism, 4 rows/s, 2.8KB/s
Parallelism: 0.0
Peak Memory: 16MB
0.24 [1 rows, 676B] [4 rows/s, 2.8KB/s]

Query 20241112_110849_00338_trhz3 failed: Could not serialize column 'a' of type 'tinyint' at position 1:1
io.trino.spi.TrinoException: Could not serialize column 'a' of type 'tinyint' at position 1:1
at io.trino.server.protocol.QueryResultRows$ResultsIterator.lambda$propagateException$1(QueryResultRows.java:424)
at java.base/java.util.Optional.ifPresent(Optional.java:178)
at io.trino.server.protocol.QueryResultRows$ResultsIterator.propagateException(QueryResultRows.java:424)
at io.trino.server.protocol.QueryResultRows$ResultsIterator.getRowValues(QueryResultRows.java:340)
at io.trino.server.protocol.QueryResultRows$ResultsIterator.computeNext(QueryResultRows.java:312)
at io.trino.server.protocol.QueryResultRows$ResultsIterator.computeNext(QueryResultRows.java:274)
at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:145)
at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:140)
at com.fasterxml.jackson.databind.ser.std.IterableSerializer.serializeContents(IterableSerializer.java:83)
at com.fasterxml.jackson.databind.ser.std.IterableSerializer.serialize(IterableSerializer.java:74)
at com.fasterxml.jackson.databind.ser.std.IterableSerializer.serialize(IterableSerializer.java:12)
at com.fasterxml.jackson.databind.ser.std.StdDelegatingSerializer.serialize(StdDelegatingSerializer.java:167)
at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:732)
at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:772)
at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:178)
at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:479)
at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:318)
at com.fasterxml.jackson.databind.ObjectWriter$Prefetch.serialize(ObjectWriter.java:1572)
at com.fasterxml.jackson.databind.ObjectWriter.writeValue(ObjectWriter.java:1061)
at io.airlift.jaxrs.JsonMapper.write(JsonMapper.java:103)
at io.airlift.jaxrs.AbstractJacksonMapper.writeTo(AbstractJacksonMapper.java:156)
at io.airlift.jaxrs.JsonMapper.writeTo(JsonMapper.java:41)
at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$TerminalWriterInterceptor.invokeWriteTo(WriterInterceptorExecutor.java:242)
at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$TerminalWriterInterceptor.aroundWriteTo(WriterInterceptorExecutor.java:227)
at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:139)
at org.glassfish.jersey.server.internal.JsonWithPaddingInterceptor.aroundWriteTo(JsonWithPaddingInterceptor.java:85)
at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:139)
at org.glassfish.jersey.server.internal.MappableExceptionWrapperInterceptor.aroundWriteTo(MappableExceptionWrapperInterceptor.java:61)
at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:139)
at org.glassfish.jersey.message.internal.MessageBodyFactory.writeTo(MessageBodyFactory.java:1116)
at org.glassfish.jersey.server.ServerRuntime$Responder.writeResponse(ServerRuntime.java:677)
at org.glassfish.jersey.server.ServerRuntime$Responder.processResponse(ServerRuntime.java:385)
at org.glassfish.jersey.server.ServerRuntime$Responder.process(ServerRuntime.java:375)
at org.glassfish.jersey.server.ServerRuntime$AsyncResponder$3.run(ServerRuntime.java:905)
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248)
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244)
at org.glassfish.jersey.internal.Errors.process(Errors.java:292)
at org.glassfish.jersey.internal.Errors.process(Errors.java:274)
at org.glassfish.jersey.internal.Errors.process(Errors.java:244)
at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265)
at org.glassfish.jersey.server.ServerRuntime$AsyncResponder.resume(ServerRuntime.java:937)
at org.glassfish.jersey.server.ServerRuntime$AsyncResponder.resume(ServerRuntime.java:893)
at io.airlift.jaxrs.AsyncResponseHandler$1.onSuccess(AsyncResponseHandler.java:99)
at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1133)
at io.airlift.concurrent.BoundedExecutor.drainQueue(BoundedExecutor.java:79)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
at java.base/java.lang.Thread.run(Thread.java:1583)
Caused by: java.lang.ClassCastException: class io.trino.spi.block.IntArrayBlock cannot be cast to class io.trino.spi.block.ByteArrayBlock (io.trino.spi.block.IntArrayBlock and io.trino.spi.block.ByteArrayBlock are in unnamed module of loader 'app')
at io.trino.spi.type.TinyintType.getByte(TinyintType.java:175)
at io.trino.spi.type.TinyintType.getObjectValue(TinyintType.java:119)
at io.trino.server.protocol.QueryResultRows$ResultsIterator.getRowValues(QueryResultRows.java:333)
... 44 more

```

### How should we improve?

_No response_

Contributor guide

Open the contributing guide

Research direction

Reproduce the AVRO table creation, insert, and select shown in the issue, then trace the failure through io.trino.server.protocol.QueryResultRows and io.trino.spi.type.TinyintType. Done means selecting data from AVRO tables with tinyint and smallint columns succeeds without the reported serialization or block-cast failure.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, sql
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.