Azure CLI connection timeout when running a long test using the ScenarioTest framework
- Dominant language
- Python
- Stars
- 4.6k
- Forks
- 3.5k
- Avg merge
- 3d 2h
- Merged PRs (30d)
- 60
Description
### Describe the bug
My team is in the process of creating a new Azure CLI extension. As part of creating this extension, we wanted to add integration tests which would test our new commands. According to this document: https://github.com/Azure/azure-cli/blob/dev/doc/authoring_tests.md in the azure-cli repository, I should be able to author integration tests by taking advantage of the ScenarioTest class.
When I run `azdev test aosm` the publish command in our extension manages to deploy a few resources but fails after a few minutes with the error attached below.
Interestingly, if I reduce the number of deployments that need to be done by this command, the command manages to reach completion. This (and the error) suggest to me that the testing framework implements some sort of timeout which I cannot modify and which does not allow my command to complete running and succeed. Is this correct? Is there anything I can do to remove this timeout condition?
### Related command
`azdev test aosm`
### Errors
```
self =
request =
kwargs = {'stream': False}, response = None
error = ServiceResponseError("('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))")
connection_timeout = 300, timeout = (300, 300), read_timeout = 300
def send(self, request, **kwargs): # type: ignore
"""Send request object according to configuration.
:param request: The request object to be sent.
:type request: ~azure.core.pipeline.transport.HttpRequest
:return: An HTTPResponse object.
:rtype: ~azure.core.pipeline.transport.HttpResponse
:keyword requests.Session session: will override the driver session and use yours.
Should NOT be done unless really required. Anything else is sent straight to requests.
:keyword dict proxies: will define the proxy to use. Proxy is a dict (protocol, url)
"""
self.open()
response = None
error = None # type: Optional[AzureErrorUnion]
try:
connection_timeout = kwargs.pop('connection_timeout', self.connection_config.timeout)
if isinstance(connection_timeout, tuple):
if 'read_timeout' in kwargs:
raise ValueError('Cannot set tuple connection_timeout and read_timeout together')
_LOGGER.warning("Tuple timeout setting is deprecated")
timeout = connection_timeout
else:
read_timeout = kwargs.pop('read_timeout', self.connection_config.read_timeout)
timeout = (connection_timeout, read_timeout)
response = self.session.request( # type: ignore
request.method,
request.url,
headers=request.headers,
data=request.data,
files=request.files,
verify=kwargs.pop('connection_verify', self.connection_config.verify),
timeout=timeout,
cert=kwargs.pop('connection_cert', self.connection_config.cert),
allow_redirects=False,
**kwargs)
response.raw.enforce_content_length = True
except (urllib3.exceptions.NewConnectionError, urllib3.exceptions.ConnectTimeoutError) as err:
error = ServiceRequestError(err, error=err)
except requests.exceptions.ReadTimeout as err:
error = ServiceResponseError(err, error=err)
except requests.exceptions.ConnectionError as err:
if err.args and isinstance(err.args[0], urllib3.exceptions.ProtocolError):
error = ServiceResponseError(err, error=err)
else:
error = ServiceRequestError(err, error=err)
except requests.exceptions.ChunkedEncodingError as err:
msg = err.__str__()
if 'IncompleteRead' in msg:
_LOGGER.warning("Incomplete download: %s", err)
error = IncompleteReadError(err, error=err)
else:
_LOGGER.warning("Unable to stream download: %s", err)
error = HttpResponseError(err, error=err)
except requests.RequestException as err:
error = ServiceRequestError(err, error=err)
if error:
> raise error
E azure.core.exceptions.ServiceResponseError: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))
../../../../.virtualenvs/az-cli-env/lib/python3.8/site-packages/azure/core/pipeline/transport/_requests_basic.py:361: ServiceResponseError
```
### Issue script & Debug output
no different than the above error message
### Expected behavior
I expect the test to pass.
### Environment Summary
azure-cli 2.48.1 *
core 2.48.1 *
telemetry 1.0.8
Extensions:
connectedk8s 1.3.5
customlocation 0.1.3
k8s-extension 1.3.6
resource-graph 2.1.0
aosm 0.1.0 (dev) /home/developer/Projects/azure-cli-extensions/src/aosm
Dependencies:
msal 1.20.0
azure-mgmt-resource 22.0.0
Python location '/home/developer/.virtualenvs/az-cli-env/bin/python'
Extensions directory '/home/developer/.azure/cliextensions'
Development extension sources:
/home/developer/Projects/azure-cli-extensions
Python (Linux) 3.8.10 (default, Mar 13 2023, 10:26:41)
[GCC 9.4.0]
Legal docs and information: aka.ms/AzureCliLegal
### Additional context
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.