locationtech / locationtech/geomesa
SparkSQL - support schemas without geometries
Nobody has claimed this yet.
- Dominant language
- Scala
- Stars
- 1.5k
- Forks
- 446
- Avg merge
- 1d 8h
- Merged PRs (30d)
- 31
Description
SparkSQL - support schemas without geometries
Trying to save a schema without a geometry field throws an NPE.
It looks like the issue is from [this|https://github.com/locationtech/geomesa/blob/geomesa_2.11-2.3.0/geomesa-spark/geomesa-spark-sql/src/main/scala/org/locationtech/geomesa/spark/GeoMesaSparkSQL.scala#L240
] geom index line. We should be able to make the variable lazy, or add an Option.map.getOrElse(-1) so that it doesn't throw an NPE if the geometry descriptor is null. (Rémi Grisot verified making it lazy worked)
I only see that variable being used in two places - one is guarded by an 'isSpatial' check, and the other is only triggered if the user passes in 'spatial' as an argument, so it's probably safe to assume there will be a valid geom in those places.
Original JIRA Issue: https://geomesa.atlassian.net/browse/GEOMESA-2629
Key: GEOMESA-2629
Type: Improvement
Priority: Major
Status: To Do
Resolution: Unresolved
Reporter: Rémi Grisot
Created: Thu, 23 May 2019 08:37:28 -0400
Updated: Tue, 14 Apr 2020 15:53:44 -0400
Comments:
557058:48912df4-0004-4f90-9915-db26684ec006 (Tue, 14 Apr 2020 15:40:02 -0400):
Rémi Grisot we’ve usually taken the position that GeoMesa is aimed at indexing geospatial data. Part of that decision involves using the GeoTools DataStore API and using SimpleFeatures as an underlying concept. SimpleFeatures always have a geometry, so that’s one reason that this issue is happening.
As such, we may close this ticket as “Won’t Fix” at some point.
557058:da7f6dfa-f0a1-468b-aec5-2cb61273feb5 (Tue, 14 Apr 2020 15:53:44 -0400):
Note that simple features don't have to have a geometry, but our support for schemas without geometries is spotty as it's not a primary use case.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in geomesa-spark/geomesa-spark-sql/src/main/scala/org/locationtech/geomesa/spark/GeoMesaSparkSQL.scala at the geom index initialization around line 240. Check how that value is used for non-spatial schemas and confirm that spatial paths still require a valid geometry. Done means saving a schema without a geometry no longer throws an NPE, while spatial behavior remains unchanged.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- scala, spark
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 38/100