spring-cloud / spring-cloud/spring-cloud-vault

HttpClient5 update packaged from Spring Boot 3.4/Spring Cloud 2024.0.0 + Envoy causing unexpected behaviours from 'protocolUpgradeEnabled' configuration

Open
#738 7 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

status: waiting-for-feedback
Dominant language
Java
Stars
291
Forks
152
Avg merge
11h 24m
Merged PRs (30d)
3

Description

Describe the bug

Spring Boot 3.4.0
Spring Cloud Dependencies 2024.0.0 (Spring Cloud Starter Vault Config 4.2.0, Spring Cloud Vault Config 4.2.0)
Spring Vault Core 3.1.2
HttpClient5 5.4.1
Istio Envoy

This is a noted issue in the Spring Boot 3.4 Release notes, where Envoy is unable to parse the HTTP/1.1 TLS upgrades which were made the default as part of the current version of HttpClient5.

image

From the ClientHttpRequestFactoryFactory class within Spring Vault Core, given that Apache HttpComponents dependency exists, it would be the default client used for constructing the ClientHttpRequestFactory that is inadvertently used by VaultTemplate to create requests to Vault.
image

However, with reference to this issue raised on Istio, the ClientHttpRequestFactory spun up by ClientHttpRequestFactoryFactory has protocolUpgradeEnabled = true by default.
image

The resolution provided in the Spring Boot Release notes to define this bean to supersede the new defaults does not seem to be working as expected, as the Spring Vault instantiation happens before the Bean is registered.

Ultimately, this causes requests from VaultTemplate -> Istio -> Vault to fail.

Are there any workarounds to this issue?

Logs:
Istio error
{"upstream_cluster":"PassthroughCluster","requested_server_name":null,"upstream_local_address":null,"bytes_sent":0,"bytes_received":0,"duration":0,"protocol":"HTTP/1.1","request_id":"redacted","start_time":"2024-12-10T08:44:40.632Z","x_forwarded_for":null,"user_agent":"Apache-HttpClient/5.4.1 (Java/21.0.3)","path":"/v1/secret/vault-test","upstream_transport_failure_reason":null,"response_flags":"-","connection_termination_details":null,"response_code_details":"upgrade_failed","downstream_remote_address":"redacted","route_name":null,"downstream_local_address":"redacted","upstream_service_time":null,"method":"GET","authority":"redacted","response_code":403,"upstream_host":null}

Vault error
{"logTimeStamp":"2024-12-10T08:13:41.025079294Z","service":"vault-test","logger":"org.springframework.vault.core.lease.SecretLeaseEventPublisher$LoggingErrorListener","level":"WARN","class":"org.springframework.boot.logging.DeferredLog","method":"logTo","line":253,"file":"DeferredLog.java","thread":"main","stackTrace":"org.springframework.vault.VaultException: Status 403 Forbidden [secret/vault-test]

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with ClientHttpRequestFactoryFactory and VaultTemplate, then compare their initialization order with the Spring Boot 3.4 release-note workaround. Reproduce the request through Istio Envoy using the versions and logs in the report, and trace where protocolUpgradeEnabled is set. Done means VaultTemplate requests succeed through Envoy without relying on an ineffective late bean registration.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, spring, spring-boot
Domain
backend, cloud, security
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.