locationtech / locationtech/geomesa
Add Parquet dependency to distributed ingest jobs
Nobody has claimed this yet.
- Dominant language
- Scala
- Stars
- 1.5k
- Forks
- 446
- Avg merge
- 1d 8h
- Merged PRs (30d)
- 31
Description
Add Parquet dependency to distributed ingest jobs
Parquet is missing from libjars.
Original JIRA Issue: https://geomesa.atlassian.net/browse/GEOMESA-2943
Key: GEOMESA-2943
Type: Improvement
Priority: Major
Status: To Do
Resolution: Unresolved
Reporter: Austin Heyne
Created: Fri, 30 Oct 2020 17:15:22 -0400
Updated: Mon, 8 Mar 2021 16:34:42 -0500
Comments:
557058:48912df4-0004-4f90-9915-db26684ec006 (Mon, 8 Mar 2021 16:34:42 -0500):
Austin Heyne hmmm… I would have expected a MapReduce environment to have Parquet available. Were you using EMR?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating the distributed ingest jobs and the libjars configuration that packages their dependencies. Confirm how Parquet is currently resolved in the MapReduce environment, then verify that the affected jobs include it in libjars and can run with the dependency available.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- scala
- Domain
- build-system, data-engineering, distributed-systems
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100