BHoM / BHoM/Test_Toolkit

Formalising Testing Framework and Approach

Open
#106 2 comments 0 reactions 4 assignees Claimed by @alelom View on GitHub
type:feature type:question
Dominant language
C#
Stars
0
Forks
1
PR merge metrics
No merged PRs in 30d

Description

Introducing a greater breakdown and categorisation of our various testing scripts and procedures to enable further scaling and automation beyond our current CI and compliance.

Below is a first list of, I think, the key categories that we need to focus on. Not all will be fully automated in the first instance and priorities will vary naturally.

Listed below for comments/input and then more granular plans to be formulated around each as needed.
Focus on Interoperability Test for 3.1 Milestone as discussed and planned.

#### I. Compliance
- Development Process Compliance Tests
- _e.g. branch naming, issue and PR template checks, label checking etc._
- Code Compliance Tests
- _Review `BHoM_Test` repo and `Compliance_Test , Coverage_Test` etc._
- _Creation of Wiki - explaining BHoM Compliance which can hyperlink to each encoded Compliance Test for clarity and transparency for all developers._

#### II. Functionality
- Data-driven Unit Tests
- _datasets required of serialised method base, input & expected output as per our_ `BH.oM.Testing.UnitTest`
- _Generation of dataset as per previous gh approach to be revisited and polished_
- Interoperability Tests
- _In the first instance - scripts testing each of the implemented BHoM_Adapter functionality_
- Issue Tests
- _As currently created to enable review of each PR._
_Issue_Tests guidelines and best practice to be developed further to enable associated Regression Tests where appropriate_
- Regression Tests
- _Ensuring fixed bugs do not reappear. Ensuring previous working features do not break for subsequent Betas_
- _Regression Tests are not necessarily a separate type of test - but keen to include as a discrete category to be be included in our process of ongoing development of tests and integrated into process of raising and successfully closing PR._
- Macro Tests
- _any combination of the above_
- Backwards Compatibility Tests
- _Tests targeting deserialisation of previous versions of BHoM._
- _Potential to include creation as part of deprecation_
- Performance tests
- _Define metric of speed and checking that subsequent PRs are not X times slower to run tests_

#### III. System
- Installer Testing
- _Works on end user machine_
- _Metrics and licencing etc. information is correct._
- _Validation that installer accurately recreates BHoM environment on `master` say_
- Environment Testing
- _Both code developer and end user (see above)_

#### IV. User Testing
- Beta Testing
- _Formalised and focused user testing as part of Beta Release process_
- _Comms around best practice raising of issues/ideas from wider user community beyond release, and with Beta in the wild. To all encourage engagement_
- Sample Testing
- _Checking key samples are all still working for each release_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.