adoptium / adoptium/infrastructure
Curl v8 breaks wget on MacOS
- Dominant language
- Python
- Stars
- 96
- Forks
- 106
- Avg merge
- 1d 23h
- Merged PRs (30d)
- 13
Description
Currently in our playbooks we install the latest Curl onto a machine if it is detected that its current version is less than 7.58.0.
test-macstadium-macos1014-x64-1 and -3 are two such machines, they now have the latest curl
```
test-macstadium-macos1014-x64-1:~ administrator$ curl --version
curl 8.0.1 (x86_64-apple-darwin18.7.0) libcurl/8.0.1 (SecureTransport) OpenSSL/1.1.1t zlib/1.2.11 brotli/1.0.9 zstd/1.5.5 libidn2/2.3.4 libssh2/1.10.0 nghttp2/1.52.0 librtmp/2.3
Release-Date: 2023-03-20
Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps mqtt pop3 pop3s rtmp rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS brotli GSS-API HSTS HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz MultiSSL NTLM NTLM_WB SPNEGO SSL threadsafe TLS-SRP UnixSockets zstd
```
Initially this broke wget
```
test-macstadium-macos1014-x64-1:~ administrator$ wget 'https://api.adoptium.net/v3/binary/latest/8/ga/mac/x64/jdk/hotspot/normal/eclipse?project=jdk'
dyld: Library not loaded: /usr/local/opt/libunistring/lib/libunistring.2.dylib
Referenced from: /usr/local/bin/wget
Reason: image not found
Abort trap: 6
```
I reinstalled wget via brew, which installed it with Openssl3
```
...
perl ./Configure --prefix=/usr/local/Cellar/openssl@3/3.1.0 --openssldir=/usr/local/etc/openssl@3 --libdir=/usr/local/Cellar/openssl@3/3.1.0/lib no-ssl3 no-ssl3-method no-zlib darwin64-x86_64-cc enable-ec_ni
==> make
==> make install MANDIR=/usr/local/Cellar/openssl@3/3.1.0/share/man MANSUFFIX=ssl
==> make test
🍺 /usr/local/Cellar/openssl@3/3.1.0: 6,487 files, 29.9MB, built in 11 minutes 55 seconds
==> Installing wget
==> ./configure --prefix=/usr/local/Cellar/wget/1.21.3_1 --sysconfdir=/usr/local/etc --with-ssl=openssl --with-libssl-prefix=/usr/local/opt/openssl@3 --disable-pcre --disable-pcre2 --without-libpsl --without-inc
==> make install
...
```
and now wget works fine. However, the `get_url` ansible module now fails to download jdks
```
TASK [adoptopenjdk_install : Download latest JDK 18 release (macOS)] ***********
FAILED - RETRYING: Download latest JDK 18 release (macOS) (3 retries left).
FAILED - RETRYING: Download latest JDK 18 release (macOS) (2 retries left).
FAILED - RETRYING: Download latest JDK 18 release (macOS) (1 retries left).
fatal: [test-macstadium-macos1014-x64-1]: FAILED! => {"attempts": 3, "changed": false, "dest": "/tmp/18-installer.pkg", "elapsed": 0, "msg": "Request failed: ", "url": "https://api.adoptium.net/v3/installer/latest/18/ga/mac/x64/jdk/hotspot/normal/eclipse?project=jdk"}
```
(failed this way before and after reinstalling wget, but not before curl was updated to 8)
I think it has to do with the version of openssl ansible is using for its `get_url` module
```
(default openssl before openssl3 through brew was installed and I symlinked it to /usr/local/bin/openssl)
test-macstadium-macos1014-x64-1:~ administrator$ /usr/bin/openssl version
LibreSSL 2.6.5
test-macstadium-macos1014-x64-1:~ administrator$ /usr/local/bin/openssl version
OpenSSL 3.1.0 14 Mar 2023 (Library: OpenSSL 3.1.0 14 Mar 2023)
```
```
test-macstadium-macos1014-x64-1:~ administrator$ /usr/bin/openssl s_client -connect api.adoptium.net:443
...
Start Time: 1683306301
Timeout : 7200 (sec)
Verify return code: 10 (certificate has expired)
```
```
est-macstadium-macos1014-x64-1:~ administrator$ /usr/local/bin/openssl s_client -connect api.adoptium.net:443
...
SSL handshake has read 4704 bytes and written 444 bytes
Verification: OK
```
Contributor guide
Research direction
Start with the infrastructure playbooks that install curl and use Ansible's get_url module, then reproduce the shown commands on the affected macOS hosts. Compare the system and Homebrew OpenSSL certificate behavior and trace the get_url failure. Done means the JDK download succeeds with certificate verification on the affected machines.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- ansible, macos
- Domain
- devops, infrastructure
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 28/100