aws / aws/aws-sdk-java-v2

NO_PROXY environment variable not handling wildcards properly

Open
#5,573 3 comments 0 reactions 0 assignees View on GitHub
cross-sdk feature-request p2 proxy
Dominant language
Java
Stars
2.6k
Forks
1k
Avg merge
2d 9h
Merged PRs (30d)
51

Description

### Upcoming End-of-Support

- [X] I acknowledge the upcoming end-of-support for AWS SDK for Java v1 was [announced](https://aws.amazon.com/blogs/developer/announcing-end-of-support-for-aws-sdk-for-java-v1-x-on-december-31-2025/), and migration to AWS SDK for Java v2 is recommended.

### Describe the bug

We are using the environment variables HTTP_PROXY, HTTPS_PROXY and NO_PROXY (as well as their lowercase variants for compatibility reasons, their values are the same) to access the internet via proxy, while connecting to internal domains without the proxy.

When using the S3 SDK we observed that wildcard entries in the `NO_PROXY` / `no_proxy` are not working as intended, as those wildcards (which are of the form `.some.domain` to include subdomains of `some.domain` like `sub.some.domain` and `more.sub.some.domain`) need a `*` prepended to work in Java, like `*.some.domain`.

### Expected Behavior

When using the SDK (we are currently using it for S3 only) we would expect that our internal S3-compatible endpoint (`s3.storage.company.internal` for the sake of this issue) is not requested via the proxy, given the following environment variables:

`HTTP_PROXY` / `HTTPS_PROXY` / `http_proxy` / `https_proxy`: `http://proxy.company.internal:3128`
`NO_PROXY` / `no_proxy`: `.company.internal,.local,127.0.0.1,localhost`

### Current Behavior

The S3 endpoint `s3.storage.company.internal` is requested via the proxy.

### Reproduction Steps

Running the following code with `NO_PROXY` set to `.company.internal,127.0.0.1,localhost` will print `software.amazon.awssdk.services.s3.model.S3Exception: (Service: S3, Status Code: 403, Request ID: null)`, due to our proxy blocking requests to internal domains with a HTTP 403. Other proxy implementations might give different errors or simply also proxy internal domains, but ours doesn't, or else we would not have spotted this issue.

The same command with `NO_PROXY` set to `*.company.internal,127.0.0.1,localhost` will print a list of our buckets, as expected.

```java
public class App {

private static final String S3_ENDPOINT = "https://s3.storage.company.internal";
private static final String S3_ACCESS_KEY = "super-secret-access-key";
private static final String S3_SECRET_KEY = "super-secret-secret-key";

public static void main(String[] args) {

S3Client s3 = createS3Client(createDefaultHttpClient());

try {
ListBucketsResponse response = s3.listBuckets();
List bucketList = response.buckets();
bucketList.forEach(bucket -> {
System.out.println("Bucket Name: " + bucket.name());
});

} catch (S3Exception e) {
System.err.println(e);
System.exit(1);
}

}

private static SdkHttpClient createDefaultHttpClient() {

final ProxyConfiguration proxyConfiguration = ProxyConfiguration.builder()
.useSystemPropertyValues(false)
.build();

return ApacheHttpClient.builder()
.proxyConfiguration(proxyConfiguration)
.build();
}

private static S3Client createS3Client(SdkHttpClient httpClient) {

final AwsCredentialsProvider credentialsProvider;

final AwsCredentials credentials = AwsBasicCredentials.create(S3_ACCESS_KEY, S3_SECRET_KEY);
credentialsProvider = StaticCredentialsProvider.create(credentials);

return S3Client.builder()
.serviceConfiguration(c -> c.pathStyleAccessEnabled(true))
.endpointOverride(URI.create(S3_ENDPOINT))
.credentialsProvider(credentialsProvider)
.region(Region.EU_CENTRAL_1)
.httpClient(httpClient)
.build();

}

}
```

### Possible Solution

Currently, the `NO_PROXY` / `no_proxy` variables are transformed such that `,` is replaced by `|`, as can be seen here: https://github.com/aws/aws-sdk-java/blob/61d73631fac8535ad70666bbce9e70a1d2cea2ca/aws-java-sdk-core/src/main/java/com/amazonaws/ClientConfiguration.java#L1129

What should also be done is that for every entry in this `|`-separated list, if the entry starts with a `.`, this should be replaced with `*.`.

Meaning: Our `NO_PROXY` / `no_proxy` value `.company.internal,.local,127.0.0.1,localhost` should be effectively converted to `*.company.internal|*.local|127.0.0.1|localhost`.

### Additional Information/Context

_No response_

### AWS Java SDK version used

2.27.20

### JDK version used

OpenJDK Runtime Environment Temurin-21.0.4+7 (build 21.0.4+7-LTS)

### Operating System and version

Windows 10 Enterprise (Build 19045.4780)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.