Set rules for gzip content encoding that's friendly to tables-in-a-bucket
- Dominant language
- HTML
- Stars
- 25
- Forks
- 14
- PR merge metrics
- No merged PRs in 30d
Description
This issue was raised in a discussion on #98.
When data is stored as (potentially paginated) Search Tables, not only can the JSON files be gzip compressed at rest, but they can be served as-is with a `content-encoding: gzip` header. This adds no burden to the web server, and minimal burden to the client receiving the data.
The vast majority of real-world HTTP clients understand `content-encoding: gzip` and transparently decompress data as it is received. However, according to the HTTP specification, we should only serve gzip-compressed data if the client requests it with an `accept-encoding: gzip` header.
If we specify that Search clients MUST be capable of dealing with `content-encoding: gzip` and MUST send an `accept-encoding: gzip` header, this will make it easy for tables-in-a-bucket Search implementations to serve compressed responses, speeding up transfers and reducing storage costs.
Notes on cloud support:
* "raw" S3 can be set up to send gzip data regardless of `accept-encoding`: https://cri.dev/posts/2013-07-16-how-to-gzip-compression-of-css-and-js-files-on-s3-with-s3cmd/
* S3 with cloudfront is adaptive based on `accept-encoding` in request: https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/ServingCompressedFiles.html
* GCS is adaptive based on `accept-encoding` in request: https://cloud.google.com/storage/docs/transcoding
* Azure storage can be set up to send gzip data regardless of `accept-encoding`: https://github.com/stefangordon/azure-storage-gzip-encoding/blob/master/README.md
Contributor guide
Research direction
Start by reading the discussion in issue #98 and the HTTP specification requirements described here, then compare the S3, CloudFront, GCS, and Azure notes. Done means the Search Tables rules clearly define client requirements for accept-encoding and content-encoding gzip, including how compressed responses should be served.
Written by the indexing model from the issue text.
Assessment
- Domain
- api, backend-api-design
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100