Support for "isolated test" flag

Definition "isolated test"

An "isolated test" is a test that triggers only the corresponding rule on the paranoia level of the rule and lower paranoia levels.

This means that an isolated test for a rule on PL1 may only trigger the corresponding rule on PL1, but it may also trigger rules on PL2 and higher.

Reasoning

The CRS Status Page depends on isolated tests. Carrying a separate list of isolated tests will lead to redundancy and an outdated list. So the information whether a test is isolated or not, should be kept together with the test.

Implementation

The information is meant to be with the test, next to the description.

Implementation variant 1: New flag next to description

  - test_title: 942100-1
    desc: "Simple SQL Injection"
    isolated: true
    stages:
      - stage:

Implementation variant 2: Create metadata array next to description, add flag to new metadata collection

  - test_title: 942100-1
    desc: "Simple SQL Injection"
    metadata:
      - isolated: true
    stages:
      - stage:

Implementation variant 3: Create metadata array, move description into new metadata collection, add flag to new metadata collection

 - test_title: 942100-1
   metadata:
     - desc: "Simple SQL Injection"
     - isolated: true
   stages:
     - stage:

CAVEAT

For the CI of the rule set it is important to make sure the tests flagged as isolated are really isolated, also when we edit rules. See definition above.

Start with the FTW test schema and the CI for the rule set, focusing on how test descriptions and metadata are represented. Define how isolated-test information is stored beside each test, then ensure the CI can verify the stated isolation rule when rules change.