locationtech / locationtech/geotrellis

FileCOGLayerWriter.write from SparkCOGExamples issue

Open
#3,492 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

doc-examples
Dominant language
Scala
Stars
1.4k
Forks
360
Avg merge
6h 57m
Merged PRs (30d)
12

Description

I desperately try to apply the example given in https://github.com/locationtech/geotrellis/blob/master/doc-examples/src/main/scala/geotrellis/doc/examples/spark/COGSparkExamples.scala, 'Having an RDD ...' with a srtm geotiff as follow:

   val conf = new SparkConf()
      .setMaster("local[*]")
      .setAppName("geotiff-tiler-spark")
      .set("spark.driver.host", "localhost")

    implicit val sc: SparkContext = new SparkContext(conf)

    val localGeoTiffPath =
      new java.io.File(new java.io.File(".").getCanonicalFile, "./data/srtm_39_03.tif").getAbsolutePath

    val localCatalogPath =
      new java.io.File(new java.io.File(".").getCanonicalFile, "./data/catalog").getAbsolutePath

    val rdd: RDD[(ProjectedExtent, Tile)] = HadoopGeoTiffRDD.spatial(new Path(localGeoTiffPath))

    val layoutScheme                                         = FloatingLayoutScheme(512)
    val (zoom: Int, metadata: TileLayerMetadata[SpatialKey]) = rdd.collectMetadata[SpatialKey](layoutScheme)
    val tilerOptions = Tiler.Options(resampleMethod = Bilinear, partitioner = Some(new HashPartitioner(rdd.partitions.length)))

    val tiledRdd                           = rdd.tileToLayout[SpatialKey](metadata, tilerOptions)
    val layerRdd: TileLayerRDD[SpatialKey] = ContextRDD(tiledRdd, metadata)

    val attributeStore = FileAttributeStore(localCatalogPath)
    val writer         = FileCOGLayerWriter(attributeStore)

    writer.write(layerName = "srtm_39_03", tiles = layerRdd, tileZoom = zoom, keyIndexMethod = ZCurveKeyIndexMethod)

But when the FileCOGLayerWriter write operation is executed, I got the following error:

Exception in thread "main" java.lang.RuntimeException: Tile Layout of layer and ZoomedLayoutScheme do not match. LayoutDefinition(Extent(10.0, 44.88000000000005, 15.120000000000015, 50.0),CellSize(8.333333333333358E-4,8.333333333333248E-4),12x12 tiles,6144x6144 pixels) != LayoutDefinition(Extent(-180.0, -89.99999, 179.99999, 89.99999),CellSize(0.70312498046875,0.3515624609375),1x1 tiles,512x512 pixels)

which corresponds to an assertion from COGLayer.scala:

if(rdd.metadata.layout != layoutScheme.levelForZoom(baseZoom).layout) {
    sys.error(s"Tile Layout of layer and ZoomedLayoutScheme do not match. ${rdd.metadata.layout} != ${layoutScheme.levelForZoom(baseZoom).layout}")
}

Any idea or explanation ?

For information, here is gdalinfo output of the srtm file:

gdalinfo srtm_39_03.tif
Driver: GTiff/GeoTIFF
Files: srtm_39_03.tif
Size is 6000, 6000
Coordinate System is:
GEOGCRS["WGS 84",
    ENSEMBLE["World Geodetic System 1984 ensemble",
        MEMBER["World Geodetic System 1984 (Transit)"],
        MEMBER["World Geodetic System 1984 (G730)"],
        MEMBER["World Geodetic System 1984 (G873)"],
        MEMBER["World Geodetic System 1984 (G1150)"],
        MEMBER["World Geodetic System 1984 (G1674)"],
        MEMBER["World Geodetic System 1984 (G1762)"],
        MEMBER["World Geodetic System 1984 (G2139)"],
        ELLIPSOID["WGS 84",6378137,298.257223563,
            LENGTHUNIT["metre",1]],
        ENSEMBLEACCURACY[2.0]],
    PRIMEM["Greenwich",0,
        ANGLEUNIT["degree",0.0174532925199433]],
    CS[ellipsoidal,2],
        AXIS["geodetic latitude (Lat)",north,
            ORDER[1],
            ANGLEUNIT["degree",0.0174532925199433]],
        AXIS["geodetic longitude (Lon)",east,
            ORDER[2],
            ANGLEUNIT["degree",0.0174532925199433]],
    USAGE[
        SCOPE["Horizontal component of 3D system."],
        AREA["World."],
        BBOX[-90,-180,90,180]],
    ID["EPSG",4326]]
Data axis to CRS axis mapping: 2,1
Origin = (10.000000000000000,50.000000000000000)
Pixel Size = (0.000833333333333,-0.000833333333333)
Metadata:
  AREA_OR_POINT=Area
Image Structure Metadata:
  INTERLEAVE=BAND
Corner Coordinates:
Upper Left  (  10.0000000,  50.0000000) ( 10d 0' 0.00"E, 50d 0' 0.00"N)
Lower Left  (  10.0000000,  45.0000000) ( 10d 0' 0.00"E, 45d 0' 0.00"N)
Upper Right (  15.0000000,  50.0000000) ( 15d 0' 0.00"E, 50d 0' 0.00"N)
Lower Right (  15.0000000,  45.0000000) ( 15d 0' 0.00"E, 45d 0' 0.00"N)
Center      (  12.5000000,  47.5000000) ( 12d30' 0.00"E, 47d30' 0.00"N)
Band 1 Block=6000x1 Type=Int16, ColorInterp=Gray
  NoData Value=-32768

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with doc-examples/src/main/scala/geotrellis/doc/examples/spark/COGSparkExamples.scala and the assertion in COGLayer.scala. Reproduce the supplied SRTM example, inspect the reported tile layouts, and determine whether the example or writer behavior is incorrect. Done means the example has a documented or verified resolution and no longer fails at write time.

Written by the indexing model from the issue text.

Assessment

Tech stack
scala
Domain
data
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.