ComplianceAsCode / ComplianceAsCode/content-test-filtering
RFE - Option to skip rules not applicable for RHEL
- Dominant language
- Python
- Stars
- 8
- Forks
- 9
- PR merge metrics
- No merged PRs in 30d
Description
When reviewing a PR which changed rules only applicable to OL, though an explicit defition of [prodtype: ol7](https://github.com/ComplianceAsCode/content/blob/master/linux_os/guide/system/software/sap_host/accounts_authorized_local_users_sidadm_orasid/rule.yml#L3), it was noted that CI tests only relevant for RHEL and Fedora were executed and failed with the following output:
```
---------Bash Remediation Logs---------
ERROR - Rule 'xccdf_org.ssgproject.content_rule_accounts_authorized_local_users_sidadm_orasid' isn't present in benchmark 'xccdf_org.ssgproject.content_benchmark_RHEL-8' in '/tmp/ssgts-ds-gzmbdvrc'
ERROR - No tests found matching the rule ID(s) 'accounts_authorized_local_users_sidadm_orasid, java_jre_insecure_prompt_locked, java_jre_enable_jws, java_jre_security_revocation_check_locked, java_jre_validation_crl_locked, java_jre_validation_ocsp, java_jre_deployment_config_mandatory, java_jre_lock_untrusted_sources, java_jre_deployment_properties_exists, java_jre_validation_crl, java_jre_disable_untrusted_sources, java_jre_validation_ocsp_locked, java_jre_accepted_sites_exists, java_jre_unsigned_applications, java_jre_unsigned_applications_locked, java_jre_security_revocation_check, java_jre_blacklist_check_locked, java_jre_enable_jws_locked, java_jre_deployment_config_properties, java_jre_blacklist_check, java_jre_disable_untrusted_sources_locked, java_jre_lock_untrusted_sources_locked, java_jre_accepted_sites_properties, java_jre_insecure_prompt, java_jre_deployment_config_exists'
WARNING - Nothing has been tested!
```
The output is correct since the rule is indeed not available for RHEL8, as in this example.
In such cases, testing this rule in RHEL is not necessary since the result will always fail.
It would be good to have an option or other mechanism to not test rules in some products when the rules are not applicable for the respective product.
References:
PR: https://github.com/ComplianceAsCode/content/pull/9398
Tests CS8: https://github.com/ComplianceAsCode/content/runs/7998438537?check_suite_focus=true
Tests CS9: https://github.com/ComplianceAsCode/content/runs/7998421987?check_suite_focus=true
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reviewing the CI tests and the linked PR and test runs to understand how rules are currently selected for RHEL and Fedora. Done means rules that are not applicable to a product can be excluded from that product's tests without producing missing-rule failures.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ci-cd, testing-qa
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100