Azure / Azure/azure-cli

Multiple simultaneous calls to "az artifacts universal download" fail if no previous calls have been made to "az artifacts universal download"

Open
#24,724 6 comments 0 reactions 0 assignees View on GitHub
act-observability-squad act-platform-engineering-squad Artifacts customer-reported DevOps needs-team-attention Service Attention
Dominant language
Python
Stars
4.6k
Forks
3.5k
Avg merge
3d 2h
Merged PRs (30d)
60

Description

### **This is autogenerated. Please review and update as needed.**

## Describe the bug
Multiple simultaneous calls to "az artifacts universal download" fail if no previous calls have been made to "az artifacts universal download". This appears to be race condition caused by each call attempting to create a new cache file and only one succeeds while the other calls all fail.

This is 100% repro on our build agents as they use a fresh image every time.

**Command Name**
`az artifacts universal download`

**Errors:**
```
EXEC : error : [WinError 183] Cannot create a file when that file already exists: 'C:\\Users\\cloudtest\\.azure-devops\\python-sdk\\cache'
Traceback (most recent call last):
File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\knack/cli.py", line 233, in invoke
File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 561, in execute
File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/__init__.py", line 507, in load_arguments
File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 318, in load_arguments
File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\knack/commands.py", line 104, in load_arguments
File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/command_operation.py", line 125, in arguments_loader
File "D:\a\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/command_operation.py", line 59, in get_op_handler
File "importlib\__init__.py", line 126, in import_module
File "", line 1050, in _gcd_import
File "", line 1027, in _find_and_load
File "", line 1006, in _find_and_load_unlocked
File "", line 688, in _load_unlocked
File "", line 883, in exec_module
File "", line 241, in _call_with_frames_removed
File "C:\Program Files\Common Files\AzureCliExtensionDirectory\azure-devops\azext_devops\dev\artifacts\universal.py", line 9, in
from azext_devops.dev.common.services import resolve_instance, resolve_instance_and_project
File "C:\Program Files\Common Files\AzureCliExtensionDirectory\azure-devops\azext_devops\dev\common\services.py", line 13, in
from azext_devops.devops_sdk.connection import Connection
File "C:\Program Files\Common Files\AzureCliExtensionDirectory\azure-devops\azext_devops\devops_sdk\connection.py", line 5, in
from ._file_cache import RESOURCE_CACHE as RESOURCE_FILE_CACHE
File "C:\Program Files\Common Files\AzureCliExtensionDirectory\azure-devops\azext_devops\devops_sdk\_file_cache.py", line 113, in
DEFAULT_CACHE_DIR = get_cache_dir()
File "C:\Program Files\Common Files\AzureCliExtensionDirectory\azure-devops\azext_devops\devops_sdk\_file_cache.py", line 108, in get_cache_dir
os.makedirs(azure_devops_cache_dir)
File "os.py", line 225, in makedirs

FileExistsError: [WinError 183] Cannot create a file when that file already exists: 'C:\\Users\\cloudtest\\.azure-devops\\python-sdk\\cache'

```

## To Reproduce:
Steps to reproduce the behavior. Note that argument values have been redacted, as they may contain sensitive information.

- _Put any pre-requisite steps here..._
- Have a script run multiple concurrent "az artifacts universal download" calls on a fresh machine (this is happening on our hosted build agents, so it is always a fresh machine). Only one of the calls will succeed while the others will fail attempting to create the python-sdk cache file.

## Expected Behavior
There is a global lock around the initial creation of the cache and when multiple concurrent calls are made they wait for the creation of the cache to complete before proceeding.

## Environment Summary
```
Windows-10-10.0.22000-SP0
Python 3.10.8
Installer: MSI

azure-cli 2.42.0

Extensions:
azure-devops 0.25.0

Dependencies:
msal 1.20.0
azure-mgmt-resource 21.1.0b1
```
## Additional Context
"az artifacts universal download" commands are embedded in some projects to bring down build dependencies and we cannot control whether these calls are made simultaneously or not as the build is parallelized.

This repros on 100% of our builds because we are using hosted build agents which are always a fresh image.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.