tests should only run when applicable for adv-sg zone in such env
- Dominant language
- Java
- Stars
- 3.1k
- Forks
- 1.4k
- Avg merge
- 6d 19h
- Merged PRs (30d)
- 32
Description
##### ISSUE TYPE
* Documentation Report
* testing/CI
##### COMPONENT NAME
~~~
marvin
~~~
##### CLOUDSTACK VERSION
~~~
4.18
~~~
##### CONFIGURATION
~~~
advanced zone with security groups.
~~~
##### OS / ENVIRONMENT
##### SUMMARY
when an advanced zone with security groups is used to test for regressions, a lot of errors occur.
an example
```
ERROR: test suite for
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/nose/suite.py", line 210, in run
self.setUp()
File "/usr/local/lib/python3.6/site-packages/nose/suite.py", line 293, in setUp
self.setupContext(ancestor)
File "/usr/local/lib/python3.6/site-packages/nose/suite.py", line 316, in setupContext
try_run(context, names)
File "/usr/local/lib/python3.6/site-packages/nose/util.py", line 471, in try_run
return func()
File "/marvin/tests/smoke/test_router_dns.py", line 92, in setUpClass
zoneid=cls.zone.id)
File "/usr/local/lib/python3.6/site-packages/marvin/lib/base.py", line 3562, in create
return Network(apiclient.createNetwork(cmd).__dict__)
File "/usr/local/lib/python3.6/site-packages/marvin/cloudstackAPI/cloudstackAPIClient.py", line 2981, in createNetwork
response = self.connection.marvinRequest(command, response_type=response, method=method)
File "/usr/local/lib/python3.6/site-packages/marvin/cloudstackConnection.py", line 381, in marvinRequest
raise e
File "/usr/local/lib/python3.6/site-packages/marvin/cloudstackConnection.py", line 376, in marvinRequest
raise self.__lastError
File "/usr/local/lib/python3.6/site-packages/marvin/cloudstackConnection.py", line 310, in __parseAndGetResponse
response_cls)
File "/usr/local/lib/python3.6/site-packages/marvin/jsonHelper.py", line 155, in getResultObj
raise cloudstackException.CloudstackAPIException(respname, errMsg)
marvin.cloudstackException.CloudstackAPIException: Execute cmd: createnetwork failed, due to: errorCode: 431, errorText:Only shared guest network can be created in security group enabled zone
```
this is not necessary as we know beforehand that these tests will fail.
Tests need to be properly tagged to not run in these environments.
##### STEPS TO REPRODUCE
~~~
~~~
##### EXPECTED RESULTS
~~~
test will not run
~~~
##### ACTUAL RESULTS
~~~
test runs and fails
~~~
Contributor guide
Research direction
Start with marvin/tests/smoke/test_router_dns.py and the TestRouterDns setup shown in the report, then inspect the other regression tests that fail in an advanced zone with security groups. Done means tests that are not applicable in this environment are skipped rather than run and fail; verify the behavior with the affected test suite.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- testing-qa
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100