mapbox / mapbox/node-pre-gyp

Can't use Alternative S3 URLs with Staging/Production Option

Open
#654 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
JavaScript
Stars
1.2k
Forks
271
Avg merge
23h 40m
Merged PRs (30d)
2

Description

## Overview

It is not possible to use alternative s3 urls (path style) and s3-compatible backends (e.g. min.io) in conjunction with the `production_host` and `staging_host` option.

## Status

- Originally `host` was defined as a string property of `binary`. The required AWS settings were automatically detected and extracted from the string.
- Pull request #533 defined `production_host` and `staging_host` as string properties of `binary` and the logic that governs which is used when. The required AWS settings were also automatically detected and extracted from the strings.
- Pull request #576 added `bucket` `region` and `s3ForcePathStyle` as string properties of the `binary` object. The trio of keys, along with `host` value, enables to explicitly set access to a bucket, bypassing automatic detection and extraction.

## Problem
`bucket` `region` and `s3ForcePathStyle` are set as properties of `binary` when in fact they are (together with the endpoint) properties of the host target. Since there can be multiple host targets (staging/production) but there can only be a single explicitly set bucket name, the alternate host option doesn't work in conjunction with the `production_host` and `staging_host` option.

## Proposed Solution

Moving forward host will be defined as an object with optional properties.

#### Example 1: (S3 Virtual Host URL)
```
{
"binary": {
"host": {
"endpoint": "https://npg-mock-bucket.s3.us-east-1.amazonaws.com",
}
}
}
```
#### Example 2: (Path Style alternative host)
```
{
"binary": {
"host": {
"endpoint": "https://play.min.io",
"bucket": "node-pre-gyp-production",
"region": "us-east-1",
"s3ForcePathStyle": true
}
}
}
```
#### Example 3: (Utilizing Staging/Production)
```
{
"binary": {
"host": {
"endpoint": "https://my-production-bucket.s3.us-east-1.amazonaws.com",
},
"staging_host": {
"endpoint": "https://play.min.io",
"bucket": "node-pre-gyp-staging",
"region": "us-east-1",
"s3ForcePathStyle": true
}
}
}
```
```
{
"binary": {
"host": {
"endpoint": "https://play.min.io",
"bucket": "node-pre-gyp-production",
"region": "us-east-1",
"s3ForcePathStyle": true
},
"staging_host": {
"endpoint": "https://play.min.io",
"bucket": "node-pre-gyp-staging",
"region": "us-east-1",
"s3ForcePathStyle": true
}
}
}
```

Note: the proposed object keys are the same as the ones passed to the AWS sdk.

## Backwards compatibility

Existing string definitions and existing keys will be supported.

#### Example 1: (S3 Virtual Host URL)
```
{
"binary": {
"host": "https://npg-mock-bucket.s3.us-east-1.amazonaws.com",
}
}
```
#### Example 2: (Alternative S3 URL)
```
{
"binary": {
"host": "https://play.min.io",
"bucket": "node-pre-gyp-staging",
"region": "us-east-1",
"s3ForcePathStyle": true
}
}
```
#### Example 3: (Utilizing Staging/Production for S3 Virtual Host URL)
```
{
"binary": {
"staging_host": "https://npg-mock-bucket-staging.s3.us-east-1.amazonaws.com",
"production_host": "https://npg-mock-bucket-staging.s3.us-east-1.amazonaws.com"
}
}
```

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the configuration handling for binary.host, staging_host, and production_host, using the proposed host-object examples as the behavioral requirements. Verify that legacy string definitions and existing bucket, region, and s3ForcePathStyle keys remain supported, and that separate production and staging S3-compatible endpoints can be selected.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, javascript, node.js
Domain
cloud, tooling
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.