Altinity / Altinity/ice

Can not import a DateTime64(9) column

Open
#38 5 comments 0 reactions 1 assignee View on GitHub

@subkanthi is already working on this.

Since Aug 18, 2025.

  • #50 by @subkanthi — closed without merging
Dominant language
Java
Stars
35
Forks
7
Avg merge
5d 12h
Merged PRs (30d)
2

Description

Repro : creating a Parquet file with a DateTime64(9) column :

clickhouse-local --query "select toDateTime64(now(9),9) as current_time format Parquet" > test.parquet
./ice insert test.test -p file://$HOME/test.parquet
2025-06-25 11:10:01 [-5-thread-1] INFO c.a.i.c.internal.cmd.Insert > file:///home/aadant/test.parquet: processing
2025-06-25 11:10:01 [-5-thread-1] INFO c.a.i.c.internal.cmd.Insert > file:///home/aadant/test.parquet: jvm: heap: init=3008M,used=16M,committed=224M,free=143599M,max=143616M, gc pressure: 100.00%(12ms), off-heap: init=7M,used=52M,committed=54M, open fds: 11
2025-06-25 11:10:01 [-5-thread-1] INFO c.a.i.c.internal.cmd.Insert > file:///home/aadant/test.parquet: copying to s3://warehouse/gw_logs/test3/data/1750867801084-06a056686f5853ecfc8ab14ec56fc10fee707ecbe9b929a3b53236991e9fee4f.parquet
2025-06-25 11:10:01 [-5-thread-1] INFO c.a.i.c.internal.cmd.Insert > file:///home/aadant/test.parquet: adding data file (copy took 0s)
2025-06-25 11:10:02 [main] INFO o.a.i.SnapshotProducer > Committed snapshot 3966130372501728897 (MergeAppend)
(.venv) [aadant@fpif1-dbal2 examples]$ ./ice insert gw_logs.test4 -p file:///home/aadant/test.parquet
2025-06-25 11:13:08 [-5-thread-1] INFO c.a.i.c.internal.cmd.Insert > file:///home/aadant/test.parquet: processing
2025-06-25 11:13:08 [-5-thread-1] INFO c.a.i.c.internal.cmd.Insert > file:///home/aadant/test.parquet: jvm: heap: init=3008M,used=16M,committed=224M,free=143599M,max=143616M, gc pressure: 100.00%(14ms), off-heap: init=7M,used=51M,committed=53M, open fds: 11
2025-06-25 11:13:08 [-5-thread-1] INFO c.a.i.c.internal.cmd.Insert > file:///home/aadant/test.parquet: copying to s3://warehouse/test/test/data/1750867988512-06a056686f5853ecfc8ab14ec56fc10fee707ecbe9b929a3b53236991e9fee4f.parquet
2025-06-25 11:13:08 [main] ERROR com.altinity.ice.cli.Main > Fatal
java.io.IOException: Error processing file(s)
	at com.altinity.ice.cli.internal.cmd.Insert.run(Insert.java:264)
	at com.altinity.ice.cli.Main.insert(Main.java:317)
	at com.altinity.ice.cli.Main.lambda$main$1(Main.java:480) [9 skipped]
	at com.altinity.ice.cli.Main.main(Main.java:487) [1 skipped]
Caused by: java.io.IOException: Error processing file:///home/aadant/test.parquet
	at com.altinity.ice.cli.internal.cmd.Insert.lambda$run$3(Insert.java:249)
 [4 skipped]
Caused by: java.lang.UnsupportedOperationException: Unsupported unit for timestamp: NANOS
	at org.apache.iceberg.data.parquet.BaseParquetReaders.timestampReader(BaseParquetReaders.java:128)
	at org.apache.iceberg.data.parquet.BaseParquetReaders$LogicalTypeReadBuilder.visit(BaseParquetReaders.java:211)
	at org.apache.parquet.schema.LogicalTypeAnnotation$TimestampLogicalTypeAnnotation.accept(LogicalTypeAnnotation.java:710)
	at org.apache.iceberg.data.parquet.BaseParquetReaders$ReadBuilder.primitive(BaseParquetReaders.java:394)
	at org.apache.iceberg.data.parquet.BaseParquetReaders$ReadBuilder.primitive(BaseParquetReaders.java:253)
	at org.apache.iceberg.parquet.TypeWithSchemaVisitor.visit(TypeWithSchemaVisitor.java:52)
	at org.apache.iceberg.parquet.TypeWithSchemaVisitor.visitField(TypeWithSchemaVisitor.java:183)
	at org.apache.iceberg.parquet.TypeWithSchemaVisitor.visitFields(TypeWithSchemaVisitor.java:198)
	at org.apache.iceberg.parquet.TypeWithSchemaVisitor.visit(TypeWithSchemaVisitor.java:47)
	at org.apache.iceberg.data.parquet.BaseParquetReaders.createReader(BaseParquetReaders.java:69)
	at org.apache.iceberg.data.parquet.BaseParquetReaders.createReader(BaseParquetReaders.java:61)
	at org.apache.iceberg.data.parquet.GenericParquetReaders.buildReader(GenericParquetReaders.java:51)
	at com.altinity.ice.cli.internal.cmd.Insert.lambda$processFile$5(Insert.java:468)
	at org.apache.iceberg.parquet.ReadConf.<init>(ReadConf.java:121)
	at org.apache.iceberg.parquet.ParquetReader.init(ParquetReader.java:71)
	at org.apache.iceberg.parquet.ParquetReader.iterator(ParquetReader.java:91)
	at org.apache.iceberg.parquet.ParquetReader.iterator(ParquetReader.java:37)
	at org.apache.iceberg.io.FileAppender.addAll(FileAppender.java:36)
	at com.altinity.ice.cli.internal.cmd.Insert.copy(Insert.java:713)
	at com.altinity.ice.cli.internal.cmd.Insert.processFile(Insert.java:478)
	at com.altinity.ice.cli.internal.cmd.Insert.lambda$run$3(Insert.java:228)
	... 4 common frames omitted

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.