aws / aws/aws-sdk-java-v2

NO_PROXY environment variable not handling wildcards properly

Aperta
#5,573 3 commenti 0 reazioni 0 assegnatari Vedi su GitHub
cross-sdk feature-request p2 proxy
Lingua principale
Java
Stelle
2.6k
Fork
1k
Merge medio
2g 9h
PR unite (30g)
51

Descrizione

### Upcoming End-of-Support

- [X] I acknowledge the upcoming end-of-support for AWS SDK for Java v1 was [announced](https://aws.amazon.com/blogs/developer/announcing-end-of-support-for-aws-sdk-for-java-v1-x-on-december-31-2025/), and migration to AWS SDK for Java v2 is recommended.

### Describe the bug

We are using the environment variables HTTP_PROXY, HTTPS_PROXY and NO_PROXY (as well as their lowercase variants for compatibility reasons, their values are the same) to access the internet via proxy, while connecting to internal domains without the proxy.

When using the S3 SDK we observed that wildcard entries in the `NO_PROXY` / `no_proxy` are not working as intended, as those wildcards (which are of the form `.some.domain` to include subdomains of `some.domain` like `sub.some.domain` and `more.sub.some.domain`) need a `*` prepended to work in Java, like `*.some.domain`.

### Expected Behavior

When using the SDK (we are currently using it for S3 only) we would expect that our internal S3-compatible endpoint (`s3.storage.company.internal` for the sake of this issue) is not requested via the proxy, given the following environment variables:

`HTTP_PROXY` / `HTTPS_PROXY` / `http_proxy` / `https_proxy`: `http://proxy.company.internal:3128`
`NO_PROXY` / `no_proxy`: `.company.internal,.local,127.0.0.1,localhost`

### Current Behavior

The S3 endpoint `s3.storage.company.internal` is requested via the proxy.

### Reproduction Steps

Running the following code with `NO_PROXY` set to `.company.internal,127.0.0.1,localhost` will print `software.amazon.awssdk.services.s3.model.S3Exception: (Service: S3, Status Code: 403, Request ID: null)`, due to our proxy blocking requests to internal domains with a HTTP 403. Other proxy implementations might give different errors or simply also proxy internal domains, but ours doesn't, or else we would not have spotted this issue.

The same command with `NO_PROXY` set to `*.company.internal,127.0.0.1,localhost` will print a list of our buckets, as expected.

```java
public class App {

private static final String S3_ENDPOINT = "https://s3.storage.company.internal";
private static final String S3_ACCESS_KEY = "super-secret-access-key";
private static final String S3_SECRET_KEY = "super-secret-secret-key";

public static void main(String[] args) {

S3Client s3 = createS3Client(createDefaultHttpClient());

try {
ListBucketsResponse response = s3.listBuckets();
List bucketList = response.buckets();
bucketList.forEach(bucket -> {
System.out.println("Bucket Name: " + bucket.name());
});

} catch (S3Exception e) {
System.err.println(e);
System.exit(1);
}

}

private static SdkHttpClient createDefaultHttpClient() {

final ProxyConfiguration proxyConfiguration = ProxyConfiguration.builder()
.useSystemPropertyValues(false)
.build();

return ApacheHttpClient.builder()
.proxyConfiguration(proxyConfiguration)
.build();
}

private static S3Client createS3Client(SdkHttpClient httpClient) {

final AwsCredentialsProvider credentialsProvider;

final AwsCredentials credentials = AwsBasicCredentials.create(S3_ACCESS_KEY, S3_SECRET_KEY);
credentialsProvider = StaticCredentialsProvider.create(credentials);

return S3Client.builder()
.serviceConfiguration(c -> c.pathStyleAccessEnabled(true))
.endpointOverride(URI.create(S3_ENDPOINT))
.credentialsProvider(credentialsProvider)
.region(Region.EU_CENTRAL_1)
.httpClient(httpClient)
.build();

}

}
```

### Possible Solution

Currently, the `NO_PROXY` / `no_proxy` variables are transformed such that `,` is replaced by `|`, as can be seen here: https://github.com/aws/aws-sdk-java/blob/61d73631fac8535ad70666bbce9e70a1d2cea2ca/aws-java-sdk-core/src/main/java/com/amazonaws/ClientConfiguration.java#L1129

What should also be done is that for every entry in this `|`-separated list, if the entry starts with a `.`, this should be replaced with `*.`.

Meaning: Our `NO_PROXY` / `no_proxy` value `.company.internal,.local,127.0.0.1,localhost` should be effectively converted to `*.company.internal|*.local|127.0.0.1|localhost`.

### Additional Information/Context

_No response_

### AWS Java SDK version used

2.27.20

### JDK version used

OpenJDK Runtime Environment Temurin-21.0.4+7 (build 21.0.4+7-LTS)

### Operating System and version

Windows 10 Enterprise (Build 19045.4780)

Guida per i contributori

Apri la guida per i contributori

Direzione di ricerca

Inizia con il parsing collegato di ClientConfiguration.java vicino alla trasformazione di NO_PROXY, quindi individua il codice di configurazione del proxy corrispondente in SDK v2. Verifica come vengono gestite le voci che iniziano con un punto e aggiungi una copertura per il pattern dell’endpoint compatibile con S3 segnalato. Il lavoro è completato quando le voci con un punto iniziale funzionano senza richiedere un prefisso '*.' esplicito e il test di regressione ha esito positivo.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
java
Ambito
networking
Tipo di issue
Bug
Difficoltà
3/5
Tempo stimato
1-2 giorni
Stato di attività
Ferma
Chiarezza
Abbastanza chiara
Idoneità per principianti
42/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.