Documentation on Java spatial RDDs
Nobody has claimed this yet.
Assessment
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Newbie friendliness
- 48/100
- Issue type
- Documentation
- Clarity
- Mostly clear
- Activity status
- Quiet
- Tech stack
- java, spark
- Domain
- data-engineering, documentation
Research direction
No documentation file or test is named. Start by locating the current GeoMesa Spark documentation for Java spatial RDDs and compare it with the supplied example and current GeoMesa version. Done means the example is verified or updated for the current version and the guidance addresses whether Spark SQL or DataFrames provide a more concise approach.
Written by the indexing model from the issue text.
Description
Documentation on Java spatial RDDs
Example from Tom Kunicki using GM 1.3.1 (should be verified for current gm version). Tom also says "If you're looking to use Spark SQL or DataFrames this sample could be made more concise".
import org.apache.hadoop.conf.Configuration; import org.apache.spark.SparkConf; import org.apache.spark.api.java.JavaSparkContext; import org.geotools.data.Query; import org.geotools.filter.text.ecql.ECQL; import org.locationtech.geomesa.spark.api.java.*; import java.util.HashMap; import java.util.Map; public class GeoMesaJavaSpark { public static void main(String... ags) throws Exception { SparkConf conf = new SparkConf() .setMaster("local[4]”) .setAppName("Sample Application"); conf.set("spark.serializer", "org.apache.spark.serializer.KryoSerializer"); conf.set("spark.kryo.registrator", "org.locationtech.geomesa.spark.GeoMesaSparkKryoRegistrator"); JavaSparkContext jsc = new JavaSparkContext(conf); MapString, String> params = new HashMap<>(); params.put("instanceId", "local"); params.put("zookeepers","localhost"); params.put("user", "root"); params.put("password", "secret"); params.put("tableName", "geomesa.gdelt"); JavaSpatialRDDProvider provider = JavaGeoMesaSpark.apply((Map)params); String filter = "BBOX(geom, -125, -24, -66, 50) AND dtg >= 2015-12-31T00:00:00Z"; Query query = new Query("gdelt", ECQL.toFilter(filter)); JavaSpatialRDD rdd = provider.rdd(new Configuration(), jsc, params, query); System.out.println(rdd.count()); System.out.println(rdd.first()); System.out.println(rdd.asGeoJSONString().first()); System.out.println(rdd.asKeyValueList().first()); System.out.println(rdd.asKeyValueMap().first()); } }
Original JIRA Issue: https://geomesa.atlassian.net/browse/GEOMESA-2192
Key: GEOMESA-2192
Type: Story
Priority: Major
Status: To Do
Resolution: Unresolved
Reporter: Emilio Lahr-Vivaz
Created: Fri, 23 Feb 2018 09:17:35 -0500
Updated: Tue, 27 Mar 2018 16:44:24 -0400
- Dominant language
- Scala
- Stars
- 1.5k
- Forks
- 446
- Avg merge
- 1d 8h
- Merged PRs (30d)
- 31
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from locationtech/geomesa
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
locationtech/geomesa#10761 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
locationtech/geomesa#10324 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
locationtech/geomesa#10111 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
locationtech/geomesa#10052 ·
-
Difficulty 1/5 1-3 hours Newbie friendliness 78/100
locationtech/geomesa#9998 ·
All issues in locationtech/geomesa
Similar issues
-
Difficulty 1/5 Under an hour Newbie friendliness 90/100
-
requires-triage
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
apache/datafusion-comet#6060 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
lichess-org/lila#21756 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 90/100
-
enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
salesforce/evalon#10 ·