Azure / Azure/azure-cli-extensions

{Package} Upgrade setuptools and wheel to the latest version.

Open
#7,740 6 comments 0 reactions 1 assignee Claimed by @aryasadeghi1 View on GitHub
Azure CLI Team feature-request
Dominant language
Python
Stars
454
Forks
1.7k
Avg merge
2d 19h
Merged PRs (30d)
64

Description

### Describe the bug
#### All users using azdev must pin wheel==0.30.0 and setuptools==70.0.0, otherwise the following issues may occur:
1. Generated index.json is incomplete, [missing metadata](https://github.com/Azure/azure-cli-extensions/pull/8556/commits/2c445334625dc763fb1a8936c24975a18ca13851). (Azdev relies on metadata generated by wheel 0.30.0 to update index.json. )
2. Error: ModuleNotFoundError: No module named 'wheel.wheelfile'
3. The wheel generated with 0.30.0 doesn't support metadata version 2.1 and cannot publish to pypi.

Example of a incomplete index.json:
```
{
"downloadUrl": "https://azuremlsdktestpypi.blob.core.windows.net/wheels/sdk-cli-v2-public/ml-2.36.0-py3-none-any.whl",
"filename": "ml-2.36.0-py3-none-any.whl",
"metadata": {
"azext.minCliCoreVersion": "2.15.0"
},
"sha256Digest": "xxx"
}
```

#### Root cause analysis:
The setuptools >=70.1.0 adds a vendored wheel 0.4.3 to generate wheel and it does not generate metadata.json.
Related PR:
https://github.com/pypa/setuptools/pull/4369
Wheel version higher than 0.30.0 has removed metadata.json from the generated files in .dist-info
https://github.com/pypa/wheel/issues/195

#### If we upgrade the setuptools to the latest version and decouple wheel will also face many issues:
1. The latest editable mode will change sys.path
2. Default build isolation behavior affecting extension installation
3. The latest editable mode will change location
4. If we generate metadata ourselves, it cannot be completely consistent with the existing metadata in idex.json.
5. Need to upgrade to pyproject.toml in the future

azure-cli-extensions:
https://github.com/Azure/azure-cli-extensions/blob/ea6fff61b28c6b87227cf657a0fbf55e03fbb863/.azure-pipelines/templates/azdev_setup.yml#L21
https://github.com/Azure/azure-cli-extensions/blob/ea6fff61b28c6b87227cf657a0fbf55e03fbb863/azure-pipelines.yml#L79
https://github.com/Azure/azure-cli-extensions/blob/ea6fff61b28c6b87227cf657a0fbf55e03fbb863/azure-pipelines.yml#L104-L105
https://github.com/Azure/azure-cli-extensions/blob/ea6fff61b28c6b87227cf657a0fbf55e03fbb863/scripts/ci/util.py#L55-L78

azure-cli:
`self.assertNotIn('metadata.json', os.listdir(os.path.join(self.ext_dir, ext_name + '-' + ext_version)))`
https://github.com/Azure/azure-cli/blob/dev/src/azure-cli-core/azure/cli/core/extension/tests/latest/test_dev_type_extension.py#L42
`self.assertIn('metadata.json', os.listdir(os.path.join(self.ext_dir, dist_info)))`
https://github.com/Azure/azure-cli/blob/dev/src/azure-cli-core/azure/cli/core/extension/tests/latest/test_wheel_type_extension.py#L46
`self.assertNotIn('metadata.json', os.listdir(os.path.join(self.ext_dir, dist_info)))`
https://github.com/Azure/azure-cli/blob/dev/src/azure-cli-core/azure/cli/core/extension/tests/latest/test_wheel_type_extension.py#L77
`self.assertNotIn('metadata.json', os.listdir(os.path.join(self.ext_dir, ext_name + '-' + ext_version)))`
https://github.com/Azure/azure-cli/blob/dev/src/azure-cli-core/azure/cli/core/extension/tests/latest/test_wheel_type_extension.py#L105
`get_cli_dependencies: whl_metadata_filepath = os.path.join(tmp_dir, dist_info_dir, 'metadata.json')`
https://github.com/Azure/azure-cli/blob/50ccb24528eb36529d1432808f3ff385cc4c04aa/tools/automation/verify/default_modules.py#L33
`pip install setuptools==70.0.0 wheel==0.30.0`
https://github.com/Azure/azure-cli/blob/bf7352f1e9d20ff04c2f7cf5dd7c1b0988ce96c1/azure-pipelines.yml#L1176

azdev:
`'wheel==0.30.0',`
https://github.com/Azure/azure-cli-dev-tools/blob/8aa045a249297d063334c89f3d70f1e84c678007/setup.py#L89
`get_ext_metadata: WHL_METADATA_FILENAME = 'metadata.json'`
https://github.com/Azure/azure-cli-dev-tools/blob/dev/azdev/operations/extensions/util.py#L48

### Related command

```
az extension add
az extension list
az extension list-available
az extension remove
az extension show
az extension update
azdev extension build
azdev extension publish
azdev extension update-index
```

### Errors

```
metadata in the wheel file is: {
"azext.minCliCoreVersion": "2.56.0",
"azext.isPreview": true
}
Traceback (most recent call last):
File "/mnt/vss/_work/1/s/scripts/ci/azdev_linter_style.py", line 290, in
main()
File "/mnt/vss/_work/1/s/scripts/ci/azdev_linter_style.py", line 281, in main
azdev_on_internal_extension(modified_files, azdev_type)
File "/mnt/vss/_work/1/s/scripts/ci/azdev_linter_style.py", line 233, in azdev_on_internal_extension
azdev_extension.check_extension_name()
File "/mnt/vss/_work/1/s/scripts/ci/azdev_linter_style.py", line 116, in check_extension_name
if metadata['name'] != extension_name:
~~~~~~~~^^^^^^^^
KeyError: 'name'
```

### Issue script & Debug output

None

### Expected behavior

None

### Environment Summary

None

### Additional context

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.