apache / apache/cloudstack

tests should only run when applicable for adv-sg zone in such env

Open
#7,155 8 comments 0 reactions 0 assignees View on GitHub
long-term-plan type:enhancement
Dominant language
Java
Stars
3.1k
Forks
1.4k
Avg merge
6d 19h
Merged PRs (30d)
32

Description

##### ISSUE TYPE

* Documentation Report
* testing/CI

##### COMPONENT NAME

~~~
marvin
~~~

##### CLOUDSTACK VERSION

~~~
4.18
~~~

##### CONFIGURATION

~~~
advanced zone with security groups.
~~~

##### OS / ENVIRONMENT

##### SUMMARY

when an advanced zone with security groups is used to test for regressions, a lot of errors occur.
an example
```
ERROR: test suite for
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/nose/suite.py", line 210, in run
self.setUp()
File "/usr/local/lib/python3.6/site-packages/nose/suite.py", line 293, in setUp
self.setupContext(ancestor)
File "/usr/local/lib/python3.6/site-packages/nose/suite.py", line 316, in setupContext
try_run(context, names)
File "/usr/local/lib/python3.6/site-packages/nose/util.py", line 471, in try_run
return func()
File "/marvin/tests/smoke/test_router_dns.py", line 92, in setUpClass
zoneid=cls.zone.id)
File "/usr/local/lib/python3.6/site-packages/marvin/lib/base.py", line 3562, in create
return Network(apiclient.createNetwork(cmd).__dict__)
File "/usr/local/lib/python3.6/site-packages/marvin/cloudstackAPI/cloudstackAPIClient.py", line 2981, in createNetwork
response = self.connection.marvinRequest(command, response_type=response, method=method)
File "/usr/local/lib/python3.6/site-packages/marvin/cloudstackConnection.py", line 381, in marvinRequest
raise e
File "/usr/local/lib/python3.6/site-packages/marvin/cloudstackConnection.py", line 376, in marvinRequest
raise self.__lastError
File "/usr/local/lib/python3.6/site-packages/marvin/cloudstackConnection.py", line 310, in __parseAndGetResponse
response_cls)
File "/usr/local/lib/python3.6/site-packages/marvin/jsonHelper.py", line 155, in getResultObj
raise cloudstackException.CloudstackAPIException(respname, errMsg)
marvin.cloudstackException.CloudstackAPIException: Execute cmd: createnetwork failed, due to: errorCode: 431, errorText:Only shared guest network can be created in security group enabled zone
```

this is not necessary as we know beforehand that these tests will fail.
Tests need to be properly tagged to not run in these environments.

##### STEPS TO REPRODUCE

~~~

~~~

##### EXPECTED RESULTS

~~~
test will not run
~~~

##### ACTUAL RESULTS

~~~
test runs and fails
~~~

Contributor guide

Open the contributing guide

Research direction

Start with marvin/tests/smoke/test_router_dns.py and the TestRouterDns setup shown in the report, then inspect the other regression tests that fail in an advanced zone with security groups. Done means tests that are not applicable in this environment are skipped rather than run and fail; verify the behavior with the affected test suite.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.