apache / apache/cloudstack

more fine grained categorisation for tests

未關閉
#4,370 6 則留言 1 個 reaction 已指派 0 人 在 GitHub 檢視
component:integration-test long-term-plan Severity:Major status:on-hold
主要語言
Java
星號
3.1k
分支
1.4k
平均合併
6 天 19 小時
30 天內合併 PR
32

描述

as a tester, one would want a more fine grained categorisation than "smoke", "component", "integration" to be able to quickly test a PR against features it touches on.

##### ISSUE TYPE

* Improvement Request
* Enhancement Request
* Feature Idea
* Other : testing aid

##### COMPONENT NAME

~~~
Marvin
~~~

##### CLOUDSTACK VERSION

~~~

~~~

##### CONFIGURATION

##### OS / ENVIRONMENT

##### SUMMARY

when a PR is tested it is usually tested against a set of smoke tests in travis or a private cloud with s.moke tests and than maybe avery feature specific (often manual) test. A better categorisation would help in testing more related code and would allow for bypassing any tests that have no baring on the subject of the change.
A blueorangutan smoke test run takes anywhere between 8 and 15 hours and fails only once after the run, albeit with multiple reports. It would be helpful to run a smaller set of basic tests. if this set passes than a more high level batch could be choosen for a next step.

##### STEPS TO REPRODUCE

~~~
go-test smoke
go-test accounts-and-roles
go-test instance-groups
go-test ...
~~~

`go-test` could be any test infra structure like blueorangutan. It is used to illustrate the categorisation, not the test-infrastructure.

##### EXPECTED RESULTS

~~~

~~~

##### ACTUAL RESULTS

~~~

~~~

貢獻指南

開啟貢獻指南

研究方向

該 issue 提到了 Travis、private-cloud smoke tests 和 blueorangutan;首先檢視現有的 smoke、component 和 integration 類別是如何定義與選取的。完成的標準是提出更細緻的類別,以支援有針對性的 PR 測試、避免無關測試,並允許從一組較小的基本測試逐步執行到更廣泛的功能專屬涵蓋範圍。

由索引模型根據 Issue 內容生成。

評估

技術堆疊
java
領域
testing-qa
Issue 類型
功能
難度
5/5
預估耗時
一週以上
活躍度
冷清
描述清晰度
需要釐清
新手友好度
35/100

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。