Documentation on Java spatial RDDs

Open
#9,466 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
48/100
Issue type
Documentation
Clarity
Mostly clear
Activity status
Quiet
Tech stack
java, spark

Research direction

No documentation file or test is named. Start by locating the current GeoMesa Spark documentation for Java spatial RDDs and compare it with the supplied example and current GeoMesa version. Done means the example is verified or updated for the current version and the guidance addresses whether Spark SQL or DataFrames provide a more concise approach.

Written by the indexing model from the issue text.

Description

Documentation on Java spatial RDDs

Example from Tom Kunicki using GM 1.3.1 (should be verified for current gm version). Tom also says "If you're looking to use Spark SQL or DataFrames this sample could be made more concise".
import org.apache.hadoop.conf.Configuration; import org.apache.spark.SparkConf; import org.apache.spark.api.java.JavaSparkContext; import org.geotools.data.Query; import org.geotools.filter.text.ecql.ECQL; import org.locationtech.geomesa.spark.api.java.*; import java.util.HashMap; import java.util.Map; public class GeoMesaJavaSpark { public static void main(String... ags) throws Exception { SparkConf conf = new SparkConf() .setMaster("local[4]”) .setAppName("Sample Application"); conf.set("spark.serializer", "org.apache.spark.serializer.KryoSerializer"); conf.set("spark.kryo.registrator", "org.locationtech.geomesa.spark.GeoMesaSparkKryoRegistrator"); JavaSparkContext jsc = new JavaSparkContext(conf); MapString, String> params = new HashMap<>(); params.put("instanceId", "local"); params.put("zookeepers","localhost"); params.put("user", "root"); params.put("password", "secret"); params.put("tableName", "geomesa.gdelt"); JavaSpatialRDDProvider provider = JavaGeoMesaSpark.apply((Map)params); String filter = "BBOX(geom, -125, -24, -66, 50) AND dtg >= 2015-12-31T00:00:00Z"; Query query = new Query("gdelt", ECQL.toFilter(filter)); JavaSpatialRDD rdd = provider.rdd(new Configuration(), jsc, params, query); System.out.println(rdd.count()); System.out.println(rdd.first()); System.out.println(rdd.asGeoJSONString().first()); System.out.println(rdd.asKeyValueList().first()); System.out.println(rdd.asKeyValueMap().first()); } }


Original JIRA Issue: https://geomesa.atlassian.net/browse/GEOMESA-2192

Key: GEOMESA-2192
Type: Story
Priority: Major
Status: To Do
Resolution: Unresolved
Reporter: Emilio Lahr-Vivaz
Created: Fri, 23 Feb 2018 09:17:35 -0500
Updated: Tue, 27 Mar 2018 16:44:24 -0400

Dominant language
Scala
Stars
1.5k
Forks
446
Avg merge
1d 8h
Merged PRs (30d)
31

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from locationtech/geomesa

All issues in locationtech/geomesa

Similar issues

More Scala issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.