Azure / Azure/azure-cli

`curl -L https://aka.ms/InstallAzureCli | bash` fails on macOS monterey 12.0.1 because of outdated virtualenv library

Open
#20,047 5 comments 0 reactions 2 assignees Claimed by @bebound View on GitHub
act-platform-engineering-squad customer-reported feature-request Installation
Dominant language
Python
Stars
4.6k
Forks
3.5k
Avg merge
3d 2h
Merged PRs (30d)
60

Description

> ### `az feedback` auto-generates most of the information requested below, as of CLI version 2.0.62

**Describe the bug**
`curl -L https://aka.ms/InstallAzureCli | bash` fails at the step of creating a virtual python environment in $HOME/lib/azure-cli

bin, lib, and include directories are created in $HOME/lib/azure-cli and various files are copied and symlinked. But then I get an error like the following that sys.prefix points to the original python3 binary location and not the new virtual environment directory in $HOME/lib/azure-cli.

```
ERROR: The executable /Users/breiter/lib/azure-cli/bin/python3 is not functioning
ERROR: It thinks sys.prefix is '/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8' (should be '/Users/breiter/lib/azure-cli')
```

**To Reproduce**
Using built-in /usr/bin/python3 and XCode 13.1 comand-line tools:

```
$ curl -L https://aka.ms/InstallAzureCli | bash
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- 0:00:02 --:--:-- 0
100 1405 100 1405 0 0 174 0 0:00:08 0:00:08 --:--:-- 295
Downloading Azure CLI install script from https://azurecliprod.blob.core.windows.net/install.py to /var/folders/sl/11rbvmp17h18fm57j0n_c4_80000gq/T/azure_cli_install_tmp_ERgSJA.
######################################################################## 100.0%
/var/folders/sl/11rbvmp17h18fm57j0n_c4_80000gq/T/azure_cli_install_tmp_ERgSJA: OK
Running install script.
-- Verifying Python version.
-- Python version 3.8.9 okay.

===> In what directory would you like to place the install? (leave blank to use '/Users/breiter/lib/azure-cli'):
-- '/Users/breiter/lib/azure-cli' is not empty and may contain a previous installation.

===> Remove this directory? (y/N): y
-- Deleted '/Users/breiter/lib/azure-cli'.
-- Creating directory '/Users/breiter/lib/azure-cli'.
-- We will install at '/Users/breiter/lib/azure-cli'.

===> In what directory would you like to place the 'az' executable? (leave blank to use '/Users/breiter/bin'):
-- The executable will be in '/Users/breiter/bin'.
-- Downloading virtualenv package from https://pypi.python.org/packages/source/v/virtualenv/virtualenv-16.7.7.tar.gz.
-- Downloaded virtualenv package to /var/folders/sl/11rbvmp17h18fm57j0n_c4_80000gq/T/tmp2ui5f98b/virtualenv-16.7.7.tar.gz.
-- Checksum of /var/folders/sl/11rbvmp17h18fm57j0n_c4_80000gq/T/tmp2ui5f98b/virtualenv-16.7.7.tar.gz OK.
-- Extracting '/var/folders/sl/11rbvmp17h18fm57j0n_c4_80000gq/T/tmp2ui5f98b/virtualenv-16.7.7.tar.gz' to '/var/folders/sl/11rbvmp17h18fm57j0n_c4_80000gq/T/tmp2ui5f98b'.
-- Executing: ['/Applications/Xcode.app/Contents/Developer/usr/bin/python3', 'virtualenv.py', '--python', '/Applications/Xcode.app/Contents/Developer/usr/bin/python3', '/Users/breiter/lib/azure-cli']
Already using interpreter /Applications/Xcode.app/Contents/Developer/usr/bin/python3
Using base prefix '/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8'
New python executable in /Users/breiter/lib/azure-cli/bin/python3
Also creating executable in /Users/breiter/lib/azure-cli/bin/python
ERROR: The executable /Users/breiter/lib/azure-cli/bin/python3 is not functioning
ERROR: It thinks sys.prefix is '/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8' (should be '/Users/breiter/lib/azure-cli')
ERROR: virtualenv is not compatible with this system or executable
Traceback (most recent call last):
File "/var/folders/sl/11rbvmp17h18fm57j0n_c4_80000gq/T/azure_cli_install_tmp_ERgSJA", line 415, in
main()
File "/var/folders/sl/11rbvmp17h18fm57j0n_c4_80000gq/T/azure_cli_install_tmp_ERgSJA", line 399, in main
create_virtualenv(tmp_dir, install_dir)
File "/var/folders/sl/11rbvmp17h18fm57j0n_c4_80000gq/T/azure_cli_install_tmp_ERgSJA", line 144, in create_virtualenv
exec_command(cmd, cwd=working_dir)
File "/var/folders/sl/11rbvmp17h18fm57j0n_c4_80000gq/T/azure_cli_install_tmp_ERgSJA", line 105, in exec_command
subprocess.check_call(command_list, cwd=cwd, env=env)
File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/subprocess.py", line 364, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/Applications/Xcode.app/Contents/Developer/usr/bin/python3', 'virtualenv.py', '--python', '/Applications/Xcode.app/Contents/Developer/usr/bin/python3', '/Users/breiter/lib/azure-cli']' returned non-zero exit status 100.
```

Using python39 installed with MacPorts.

```
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- 0:00:01 --:--:-- 0
100 1405 100 1405 0 0 369 0 0:00:03 0:00:03 --:--:-- 897
Downloading Azure CLI install script from https://azurecliprod.blob.core.windows.net/install.py to /var/folders/sl/11rbvmp17h18fm57j0n_c4_80000gq/T/azure_cli_install_tmp_GhQktk.
######################################################################## 100.0%
/var/folders/sl/11rbvmp17h18fm57j0n_c4_80000gq/T/azure_cli_install_tmp_GhQktk: OK
Running install script.
-- Verifying Python version.
-- Python version 3.9.7 okay.

===> In what directory would you like to place the install? (leave blank to use '/Users/breiter/lib/azure-cli'):
-- '/Users/breiter/lib/azure-cli' is not empty and may contain a previous installation.

===> Remove this directory? (y/N): y
-- Deleted '/Users/breiter/lib/azure-cli'.
-- Creating directory '/Users/breiter/lib/azure-cli'.
-- We will install at '/Users/breiter/lib/azure-cli'.

===> In what directory would you like to place the 'az' executable? (leave blank to use '/Users/breiter/bin'):
-- The executable will be in '/Users/breiter/bin'.
-- Downloading virtualenv package from https://pypi.python.org/packages/source/v/virtualenv/virtualenv-16.7.7.tar.gz.
-- Downloaded virtualenv package to /var/folders/sl/11rbvmp17h18fm57j0n_c4_80000gq/T/tmpszmiyjd7/virtualenv-16.7.7.tar.gz.
-- Checksum of /var/folders/sl/11rbvmp17h18fm57j0n_c4_80000gq/T/tmpszmiyjd7/virtualenv-16.7.7.tar.gz OK.
-- Extracting '/var/folders/sl/11rbvmp17h18fm57j0n_c4_80000gq/T/tmpszmiyjd7/virtualenv-16.7.7.tar.gz' to '/var/folders/sl/11rbvmp17h18fm57j0n_c4_80000gq/T/tmpszmiyjd7'.
-- Executing: ['/opt/local/bin/python3', 'virtualenv.py', '--python', '/opt/local/bin/python3', '/Users/breiter/lib/azure-cli']
Already using interpreter /opt/local/bin/python3
Using base prefix '/opt/local/Library/Frameworks/Python.framework/Versions/3.9'
New python executable in /Users/breiter/lib/azure-cli/bin/python3
Also creating executable in /Users/breiter/lib/azure-cli/bin/python
ERROR: The executable /Users/breiter/lib/azure-cli/bin/python3 is not functioning
ERROR: It thinks sys.prefix is '/opt/local/Library/Frameworks/Python.framework/Versions/3.9' (should be '/Users/breiter/lib/azure-cli')
ERROR: virtualenv is not compatible with this system or executable
Traceback (most recent call last):
File "/var/folders/sl/11rbvmp17h18fm57j0n_c4_80000gq/T/azure_cli_install_tmp_GhQktk", line 415, in
main()
File "/var/folders/sl/11rbvmp17h18fm57j0n_c4_80000gq/T/azure_cli_install_tmp_GhQktk", line 399, in main
create_virtualenv(tmp_dir, install_dir)
File "/var/folders/sl/11rbvmp17h18fm57j0n_c4_80000gq/T/azure_cli_install_tmp_GhQktk", line 144, in create_virtualenv
exec_command(cmd, cwd=working_dir)
File "/var/folders/sl/11rbvmp17h18fm57j0n_c4_80000gq/T/azure_cli_install_tmp_GhQktk", line 105, in exec_command
subprocess.check_call(command_list, cwd=cwd, env=env)
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/subprocess.py", line 373, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/opt/local/bin/python3', 'virtualenv.py', '--python', '/opt/local/bin/python3', '/Users/breiter/lib/azure-cli']' returned non-zero exit status 100.
```

**Expected behavior**
The install script should run to completion and result in a functioning `az` binary.

**Environment summary**
macOS monterey 12.0.1 x86_64
GNU bash, version 5.1.8(1)-release (x86_64-apple-darwin21.1.0)
python 3.8.9 (apple) or python 3.9.7 (macports)

**Additional context**
I think the issue is that virtualenv 16.x is not compatible with macOS 12. I tried modifying install.py to use 16.7.12 and that also failed. I then installed `py39-virtualenv` (which is 20.9.0) from macports and modified install.py to use `virtualenv` from the path rather than downloading it from pypi.

```dff
$ diff -u install.py.ori install.py
--- install.py.ori 2021-10-26 15:30:52.000000000 +0200
+++ install.py 2021-10-26 15:32:29.000000000 +0200
@@ -125,23 +125,24 @@


def create_virtualenv(tmp_dir, install_dir):
- download_location = os.path.join(tmp_dir, VIRTUALENV_ARCHIVE)
- print_status('Downloading virtualenv package from {}.'.format(VIRTUALENV_DOWNLOAD_URL))
- response = urlopen(VIRTUALENV_DOWNLOAD_URL)
- with open(download_location, 'wb') as f: f.write(response.read())
- print_status("Downloaded virtualenv package to {}.".format(download_location))
- if is_valid_sha256sum(download_location, VIRTUALENV_ARCHIVE_SHA256):
- print_status("Checksum of {} OK.".format(download_location))
- else:
- raise CLIInstallError("The checksum of the downloaded virtualenv package does not match.")
- print_status("Extracting '{}' to '{}'.".format(download_location, tmp_dir))
- package_tar = tarfile.open(download_location)
- package_tar.extractall(path=tmp_dir)
- package_tar.close()
- virtualenv_dir_name = 'virtualenv-'+VIRTUALENV_VERSION
- working_dir = os.path.join(tmp_dir, virtualenv_dir_name)
- cmd = [sys.executable, 'virtualenv.py', '--python', sys.executable, install_dir]
- exec_command(cmd, cwd=working_dir)
+ #download_location = os.path.join(tmp_dir, VIRTUALENV_ARCHIVE)
+ #print_status('Downloading virtualenv package from {}.'.format(VIRTUALENV_DOWNLOAD_URL))
+ #response = urlopen(VIRTUALENV_DOWNLOAD_URL)
+ #with open(download_location, 'wb') as f: f.write(response.read())
+ #print_status("Downloaded virtualenv package to {}.".format(download_location))
+ #if is_valid_sha256sum(download_location, VIRTUALENV_ARCHIVE_SHA256):
+ # print_status("Checksum of {} OK.".format(download_location))
+ #else:
+ # raise CLIInstallError("The checksum of the downloaded virtualenv package does not match.")
+ #print_status("Extracting '{}' to '{}'.".format(download_location, tmp_dir))
+ #package_tar = tarfile.open(download_location)
+ #package_tar.extractall(path=tmp_dir)
+ #package_tar.close()
+ #virtualenv_dir_name = 'virtualenv-'+VIRTUALENV_VERSION
+ #working_dir = os.path.join(tmp_dir, virtualenv_dir_name)
+ cmd = ['virtualenv', '--python', sys.executable, install_dir]
+ #exec_command(cmd, cwd=working_dir)
+ exec_command(cmd)


def install_cli(install_dir, tmp_dir):
```

This worked and ran to completion.

```
$ az --version
azure-cli 2.29.1

core 2.29.1
telemetry 1.0.6

Python location '/Users/breiter/lib/azure-cli/bin/python'
Extensions directory '/Users/breiter/.azure/cliextensions'

Python (Darwin) 3.9.7 (default, Oct 26 2021, 00:27:12)
[Clang 12.0.0 (clang-1200.0.32.28)]

Legal docs and information: aka.ms/AzureCliLegal

Your CLI is up-to-date.

Please let us know how we are doing: https://aka.ms/azureclihats
and let us know if you're interested in trying out our newest features: https://aka.ms/CLIUXstudy
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.