locationtech / locationtech/geomesa

Support official arrow file extensions

Open
#6,922 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Scala
Stars
1.5k
Forks
446
Avg merge
1d 8h
Merged PRs (30d)
31

Description

Support official arrow file extensions

Arrow has officially registered a mime type and file extensions of .arrow for the ipc file format and .arrows for ipc streaming format. We should support the mime type in geoserver requests, set it in files we create where appropriate, and support files named .arrows as well as our current .arrow in the geomesa-arrow CLI.

Relevant links:
https://www.iana.org/assignments/media-types/application/vnd.apache.arrow.file
https://www.iana.org/assignments/media-types/application/vnd.apache.arrow.stream
https://arrow.apache.org/docs/format/Columnar.html#ipc-file-format
https://arrow.apache.org/docs/format/Columnar.html#ipc-streaming-format


Original JIRA Issue: https://geomesa.atlassian.net/browse/GEOMESA-3096

Key: GEOMESA-3096
Type: Story
Priority: Major
Status: To Do
Resolution: Unresolved
Reporter: Emilio Lahr-Vivaz
Created: Tue, 6 Jul 2021 07:45:45 -0400
Updated: Tue, 6 Jul 2021 07:48:16 -0400

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing MIME handling in GeoServer requests and extension handling in the geomesa-arrow CLI. Review the official Arrow file and streaming MIME types linked in the issue, then verify that created files use the appropriate type and that both .arrow and .arrows names are supported.

Written by the indexing model from the issue text.

Assessment

Tech stack
scala
Domain
api, cli
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.