adoptium / adoptium/infrastructure

Curl v8 breaks wget on MacOS

Open
#3,048 0 comments 0 reactions 0 assignees View on GitHub
ansible os:macOS
Dominant language
Python
Stars
96
Forks
106
Avg merge
1d 23h
Merged PRs (30d)
13

Description

Currently in our playbooks we install the latest Curl onto a machine if it is detected that its current version is less than 7.58.0.

test-macstadium-macos1014-x64-1 and -3 are two such machines, they now have the latest curl
```
test-macstadium-macos1014-x64-1:~ administrator$ curl --version
curl 8.0.1 (x86_64-apple-darwin18.7.0) libcurl/8.0.1 (SecureTransport) OpenSSL/1.1.1t zlib/1.2.11 brotli/1.0.9 zstd/1.5.5 libidn2/2.3.4 libssh2/1.10.0 nghttp2/1.52.0 librtmp/2.3
Release-Date: 2023-03-20
Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps mqtt pop3 pop3s rtmp rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS brotli GSS-API HSTS HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz MultiSSL NTLM NTLM_WB SPNEGO SSL threadsafe TLS-SRP UnixSockets zstd
```
Initially this broke wget
```
test-macstadium-macos1014-x64-1:~ administrator$ wget 'https://api.adoptium.net/v3/binary/latest/8/ga/mac/x64/jdk/hotspot/normal/eclipse?project=jdk'
dyld: Library not loaded: /usr/local/opt/libunistring/lib/libunistring.2.dylib
Referenced from: /usr/local/bin/wget
Reason: image not found
Abort trap: 6
```
I reinstalled wget via brew, which installed it with Openssl3
```
...
perl ./Configure --prefix=/usr/local/Cellar/openssl@3/3.1.0 --openssldir=/usr/local/etc/openssl@3 --libdir=/usr/local/Cellar/openssl@3/3.1.0/lib no-ssl3 no-ssl3-method no-zlib darwin64-x86_64-cc enable-ec_ni
==> make

==> make install MANDIR=/usr/local/Cellar/openssl@3/3.1.0/share/man MANSUFFIX=ssl
==> make test

🍺 /usr/local/Cellar/openssl@3/3.1.0: 6,487 files, 29.9MB, built in 11 minutes 55 seconds
==> Installing wget
==> ./configure --prefix=/usr/local/Cellar/wget/1.21.3_1 --sysconfdir=/usr/local/etc --with-ssl=openssl --with-libssl-prefix=/usr/local/opt/openssl@3 --disable-pcre --disable-pcre2 --without-libpsl --without-inc
==> make install
...
```
and now wget works fine. However, the `get_url` ansible module now fails to download jdks

```
TASK [adoptopenjdk_install : Download latest JDK 18 release (macOS)] ***********
FAILED - RETRYING: Download latest JDK 18 release (macOS) (3 retries left).
FAILED - RETRYING: Download latest JDK 18 release (macOS) (2 retries left).
FAILED - RETRYING: Download latest JDK 18 release (macOS) (1 retries left).
fatal: [test-macstadium-macos1014-x64-1]: FAILED! => {"attempts": 3, "changed": false, "dest": "/tmp/18-installer.pkg", "elapsed": 0, "msg": "Request failed: ", "url": "https://api.adoptium.net/v3/installer/latest/18/ga/mac/x64/jdk/hotspot/normal/eclipse?project=jdk"}
```
(failed this way before and after reinstalling wget, but not before curl was updated to 8)

I think it has to do with the version of openssl ansible is using for its `get_url` module

```
(default openssl before openssl3 through brew was installed and I symlinked it to /usr/local/bin/openssl)
test-macstadium-macos1014-x64-1:~ administrator$ /usr/bin/openssl version
LibreSSL 2.6.5

test-macstadium-macos1014-x64-1:~ administrator$ /usr/local/bin/openssl version
OpenSSL 3.1.0 14 Mar 2023 (Library: OpenSSL 3.1.0 14 Mar 2023)
```

```
test-macstadium-macos1014-x64-1:~ administrator$ /usr/bin/openssl s_client -connect api.adoptium.net:443
...

Start Time: 1683306301
Timeout : 7200 (sec)
Verify return code: 10 (certificate has expired)
```

```
est-macstadium-macos1014-x64-1:~ administrator$ /usr/local/bin/openssl s_client -connect api.adoptium.net:443
...
SSL handshake has read 4704 bytes and written 444 bytes
Verification: OK
```

Contributor guide

Open the contributing guide

Research direction

Start with the infrastructure playbooks that install curl and use Ansible's get_url module, then reproduce the shown commands on the affected macOS hosts. Compare the system and Homebrew OpenSSL certificate behavior and trace the get_url failure. Done means the JDK download succeeds with certificate verification on the affected machines.

Written by the indexing model from the issue text.

Assessment

Tech stack
ansible, macos
Domain
devops, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.