apache / apache/sedona

Read Geotiff: java.lang.RuntimeException: javax.imageio.IIOException: 64-bit samples are not supported for Horizontal differencing Predictor

Open
#968 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
2.4k
Forks
784
Avg merge
1d 12h
Merged PRs (30d)
58

Description

## Expected behavior

Geotiff can read without "java.lang.RuntimeException: javax.imageio.IIOException: 64-bit samples are not supported for Horizontal differencing Predictor".

## Actual behavior

I receive for certain geotiff files "java.lang.RuntimeException: javax.imageio.IIOException: 64-bit samples are not supported for Horizontal differencing Predictor".

I digged a bit into this - it seems because the geotiff file is compressed (DEFLATE) using QGIS (GDAL).
However, from the error I get that certain predictor settings for deflate are not supported (cf. also: https://kokoalberti.com/articles/geotiff-compression-optimization-guide/). I still have to try using no predictor. It may work then (I have to check the QGIS GDAL settings if they use by default horizontal predictor).

I looked also in the Sedona code and in theory it should support it ([gc-geotiff dependency
](https://github.com/geotools/geotools/tree/main/modules/plugin/geotiff)). It is a bit more difficult to troubleshoot, because this depends on javax.media.jai_imageio and it.geosolutions.imageio-ext.imageio-ext-tiff.

It seems javax.media.jai_imageio in principle supports tiff deflate compression (https://github.com/jai-imageio/jai-imageio-core/blob/master/src/main/java/com/github/jaiimageio/plugins/tiff/BaselineTIFFTagSet.java#L249).

However, it.geosolutions.imageio-ext.imageio-ext-tiff (https://github.com/geosolutions-it/imageio-ext/tree/master/plugin/tiff) has an optimized tiff format reader/writer which may support less.

The "javax.media.jai_imageio" is a bit special due to its ["special history"](https://github.com/jai-imageio/jai-imageio-core) and also different variant exists (e.g. https://github.com/senbox-org/jai-imageio-core/blob/master/src/share/classes/com/sun/media/imageioimpl/plugins/tiff/TIFFDeflateDecompressor.java).

I saw that it seems to be also integrated back in JDK9 ("[Note that most of these formats are now supported by Java 9](https://github.com/jai-imageio/jai-imageio-core/blob/master/README.md)")

However gc-geotools seem to use the maven package (cf. https://github.com/geotools/geotools/blob/main/modules/plugin/geotiff/pom.xml#L105), which is based on an outdated version (https://github.com/jai-imageio/jai-imageio-core). Nevertheless it may not make sense to use the JDK version, because it is in the JDK module "java.desktop", which may not be available on server versions. Probably that is why the extensions it.geosolutions.imageio-ext.imageio-ext-tiff are included in Sedona?

I know this is probably not a Sedona problem, but a problem that Sedona users may face - so I can propose to add a documentation page in Sedona to reflect this (which compressions are supported etc.).

## Steps to reproduce the problem
I took a small file from https://www.ipcc-data.org/ to reproduce the problem. The original file "ipcc_eu_CDD_v_2040_r45_a.tiff" is not compressed and works perfectly fine with Sedona.

The other file is a cropped version of the original file of the Austria region but with DEFLATE compression using standard QGIS GDAL settings (probably horizontal predictor is default in QGIS (3.28.1), I have to check, still have to see if this works if I use DEFLATE with no predictor).

[ipcc_.tar.gz](https://github.com/apache/sedona/files/12339420/ipcc_.tar.gz)

You can reproduce it easily by using the example provided in the documentation for [RS_Values](https://sedona.apache.org/1.4.1/api/sql/Raster-operators/#rs_values), which uses [RS_FromGeoTiff
](https://sedona.apache.org/1.4.1/api/sql/Raster-loader/#rs_fromgeotiff)

## Settings

Sedona version = 1.4.1

Apache Spark version = 3.2.0

Apache Flink version = N/A

API type = Python?

Scala version =2.12

JRE version = 1.8

Python version = 3.9

Environment = CDP

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the failure with the attached ipcc_.tar.gz file using the RS_FromGeoTiff and RS_Values examples. Inspect the GeoTIFF reader dependencies mentioned in the issue, including gc-geotiff, jai-imageio, and imageio-ext-tiff. Done means the compressed file reads successfully, or the supported compression and predictor limitations are documented.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
data
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.