ComplianceAsCode / ComplianceAsCode/content
Rule configure_crypto_policy is failing tests
- Dominant language
- Shell
- Stars
- 2.8k
- Forks
- 828
- Avg merge
- 3d 8m
- Merged PRs (30d)
- 80
Description
#### Description of problem:
During the review of the productization test run test `/CoreOS/scap-security-guide/Sanity/test-rules-scenarios-per-profile OSPP 4/5` we discovered that the rule `configure_crypto_policy` fails the `cis_l2.pass.sh` test scenario for both Ansible and Bash remediations when Automatus is executed in a `combined` mode.
#### SCAP Security Guide Version:
current upstream master branch as of 2023-07-22 as of HEAD a96ccb9
#### Operating System Version:
RHEL 9
#### Steps to Reproduce:
1. python3 /tmp/tmp.GwTs8FwB4g/rpmbuild/BUILD/scap-security-guide-0.1.69/tests/test_suite.py combined --slice 5 5 --libvirt qemu:///system test_suite_vm --datastream /usr/share/xml/scap/ssg/content/ssg-rhel9-ds.xml --mode online --remediate-using bash --duplicate-templates --no-reports xccdf_org.ssgproject.content_profile_ospp
2. python3 /tmp/tmp.GwTs8FwB4g/rpmbuild/BUILD/scap-security-guide-0.1.69/tests/test_suite.py combined --slice 5 5 --libvirt qemu:///system test_suite_vm --datastream /usr/share/xml/scap/ssg/content/ssg-rhel9-ds.xml --mode online --remediate-using ansible --duplicate-templates --no-reports xccdf_org.ssgproject.content_profile_ospp
#### Actual Results:
```
INFO - xccdf_org.ssgproject.content_rule_configure_crypto_policy
INFO - Script cis_l2.pass.sh using profile xccdf_org.ssgproject.content_profile_cis OK
ERROR - Script cis_l2.pass.sh using profile xccdf_org.ssgproject.content_profile_cis_workstation_l2 found issue:
ERROR - Rule xccdf_org.ssgproject.content_rule_configure_crypto_policy has not been evaluated! Wrong profile selected in test scenario?
ERROR - The initial scan failed for rule 'xccdf_org.ssgproject.content_rule_configure_crypto_policy'.
INFO - Script config_and_current_same_time.pass.sh using profile (all) OK
INFO - Script config_newer_than_current.fail.sh using profile (all) OK
INFO - Script missing_nss_config.fail.sh using profile xccdf_org.ssgproject.content_profile_ospp OK
INFO - Script missing_policy.fail.sh using profile xccdf_org.ssgproject.content_profile_ospp OK
INFO - Script missing_policy_file.fail.sh using profile xccdf_org.ssgproject.content_profile_ospp OK
INFO - Script nss_config_as_file.pass.sh using profile xccdf_org.ssgproject.content_profile_ospp OK
INFO - Script nss_config_as_symlink.pass.sh using profile xccdf_org.ssgproject.content_profile_ospp OK
INFO - Script policy_default_cis_l1.pass.sh using profile xccdf_org.ssgproject.content_profile_cis_server_l1 OK
INFO - Script policy_default_cis_l1.pass.sh using profile xccdf_org.ssgproject.content_profile_cis_workstation_l1 OK
INFO - Script policy_default_nosha1_set.pass.sh using profile xccdf_org.ssgproject.content_profile_e8 OK
WARNING - Script policy_default_set.pass.sh - profile xccdf_org.ssgproject.content_profile_standard not found in datastream
INFO - Script policy_fips_ospp_set.pass.sh using profile xccdf_org.ssgproject.content_profile_ospp OK
INFO - Script wrong_policy.fail.sh using profile xccdf_org.ssgproject.content_profile_ospp OK
```
#### Expected Results:
no errors are reported by Automatus
#### Additional Information/Debugging Steps:
I was also able to reproduce the problem in `rule` mode locally with a RHEL 9 VM back end.
```
[jcerny@fedora scap-security-guide{master}]$ tests/automatus.py rule --libvirt qemu:///system ssgts_rhel9 --scenario cis_l2.pass.sh configure_crypto_policy
Setting console output to log level INFO
INFO - The base image option has not been specified, choosing libvirt-based test environment.
INFO - Logging into /home/jcerny/work/git/scap-security-guide/logs/rule-custom-2023-07-24-1537/test_suite.log
INFO - xccdf_org.ssgproject.content_rule_configure_crypto_policy
INFO - Script cis_l2.pass.sh using profile xccdf_org.ssgproject.content_profile_cis OK
ERROR - Script cis_l2.pass.sh using profile xccdf_org.ssgproject.content_profile_cis_workstation_l2 found issue:
ERROR - Rule xccdf_org.ssgproject.content_rule_configure_crypto_policy has not been evaluated! Wrong profile selected in test scenario?
ERROR - The initial scan failed for rule 'xccdf_org.ssgproject.content_rule_configure_crypto_policy'.
```
Also, please check if the Automatus's output isn't misleading in this case. We had a similar case where the Automatus's output was misleading: https://github.com/ComplianceAsCode/content/issues/10823
Contributor guide
Assessment
This issue has not been assessed yet.