simp / simp/rubygem-simp-compliance_engine

sce-schema.json: patternProperties without additionalProperties:false makes a malformed key skip validation entirely

Open
#136 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Ruby
Stars
2
Forks
1
Avg merge
5d 16h
Merged PRs (30d)
2

Description

properties.profiles (line 107), .ce (193), .checks (273), and .controls (421) each declare a key pattern via patternProperties but do not set additionalProperties: false. In the whole schema, additionalProperties: false appears only at the document root (line 11) and in $defs/controlsMap (line 61).

Under JSON Schema, a key that fails patternProperties and has no additionalProperties constraint is simply unconstrained. So a malformed key does not produce an error — it makes the schema skip the entire entry underneath it.

Reproduction

version: 2.0.0
checks:
  "check one":            # space in key -> no pattern match
    ces: "not-an-array"   # 0 violations: entry never validated

Rename the key and the same body validates correctly:

checks:
  check_one:
    ces: "not-an-array"   # 1 violation: expected array

Confirmed with json_schemer 2.5.0 against compliance_engine 1.1.0. A four-key fixture went from 2 violations to 4 purely by making the keys pattern-conformant — the two extra findings (ces typed as String, ce.*.title typed as Integer) were invisible while the keys were malformed.

Why it matters

This inverts the intent twice over:

  1. Key-name validation is effectively inert. Nothing enforces the naming pattern for profiles, CEs, checks, or controls.
  2. Worse, a bad key reduces coverage. The documents most likely to contain a typo'd key are exactly the ones that get the least validation.

Key-name validation is also one of the few structural checks scelint has no coverage of at all, so this is precisely where the schema would be adding something rather than duplicating.

Suggested fix

Add additionalProperties: false alongside patternProperties on all four top-level collections.

Blocked on #135 — several keys in live SCE data (profiles.*.id, profiles.*.benchmark_version, ce.*.notes, checks.*.oval-ids) are missing from the schema's properties lists and are tolerated today only because additionalProperties is open. Tightening this without adding those first would reject nearly all production data.

Note that #134 (the controlsMap key pattern) shows the failure mode on the other side: where additionalProperties: false is set, the accompanying pattern turns out to be too narrow for real identifiers. Both patterns deserve a check against real data before this is tightened.

Found while evaluating whether scelint should consume ComplianceEngine.schema as a lint pass.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with sce-schema.json and review #135 before changing the four collections named in the issue: profiles, ce, checks, and controls. Compare their key patterns and missing properties against live SCE data, including the cases noted in the issue. Done means malformed keys produce validation errors without rejecting currently supported production data, with the reproduction confirming nested values are still checked.

Written by the indexing model from the issue text.

Assessment

Tech stack
json, ruby
Domain
testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.