[R] Cannot override environmental variables setting AWS S3 region
- Dominant language
- C++
- Stars
- 17.1k
- Forks
- 4.3k
- Avg merge
- 3d 13h
- Merged PRs (30d)
- 88
Description
If a user has set the AWS_DEFAULT_REGION, there is no way we can override that setting (especially to a null value) in a one-off call. Consider the following example. This fails:
```java
library(arrow)
Sys.setenv(AWS_DEFAULT_REGION="data")
noaa <- s3_bucket("neon4cast-drivers/",
endpoint_override = "data.ecoforecast.org",
anonymous = TRUE)
```
If the env var is not set or unset, this succeeds. However, attempting to override the region does not help:
```java
noaa <- s3_bucket("neon4cast-drivers/",
endpoint_override = "data.ecoforecast.org",
region = "us-east-1",
anonymous = TRUE)
```
(nor does it help to set the region to "", or NULL. – note this is a MINIO host and so region is not needed anyway).
Relatedly, one might expect that AWS_S3_ENDPOINT could be used instead of setting `endpoint_override`, but this does not seem to work either (lemme know if you want that in a separate issue thread). At least on the plus side, the above code does not fail if AWS_S3_ENDPOINT is set to some other value than given in endpoint_override, so that is nice).
**Reporter**: [Carl Boettiger](https://issues.apache.org/jira/browse/ARROW-17201) / @cboettig
**Note**: *This issue was originally created as [ARROW-17201](https://issues.apache.org/jira/browse/ARROW-17201). Please see the [migration documentation](https://github.com/apache/arrow/issues/14542) for further details.*
Contributor guide
Research direction
Start by reproducing the R examples with AWS_DEFAULT_REGION set, then trace how endpoint_override, region, and environment variables are handled for S3 connections. Done means a one-off call can override or clear the environment-provided region when using the shown MINIO endpoint, with the reported examples succeeding.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, r
- Domain
- cloud
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100