FSDS - Use S3 select for optimized parquet reads from S3

FSDS - Use S3 select for optimized parquet reads from S3

S3 select will let us only return the columns we want, and may also support parquet predicates? It will only return data as csv or json, so we would need to run the response through a converter or something.

It's a somewhat specialized optimization, as it will only work with parquet files stored in s3, but it could save a lot of network and cost so it seems worthwhile. (technically we could also use it with 'converter' mode FSDS if the files are CSV or JSON, but that's probably not a primary use case worth pursuing.)

https://docs.aws.amazon.com/AmazonS3/latest/dev/SelectObjectContentUsingJava.html

Original JIRA Issue: https://geomesa.atlassian.net/browse/GEOMESA-2701

Key: GEOMESA-2701
Type: Story
Priority: Major
Status: To Do
Resolution: Unresolved
Reporter: Emilio Lahr-Vivaz
Created: Fri, 30 Aug 2019 10:58:19 -0400
Updated: Fri, 30 Aug 2019 10:58:19 -0400

No files, tests, or GeoMesa entry points are identified. Start with the linked Amazon S3 Select documentation and locate the FSDS S3 parquet-read path; done would require a defined approach for column or predicate selection, conversion of S3 Select responses, and coverage for the supported parquet-on-S3 use case.