Card reference lists (CAPEC/ASVS/MASTG) render out of order and with duplicates when source YAML isn't pre-sorted
- Dominant language
- Python
- Stars
- 146
- Forks
- 97
- Avg merge
- 1d 7h
- Merged PRs (30d)
- 104
Description
### Describe the bug
`group_number_ranges()` in `scripts/convert.py` compresses a list of reference numbers (CAPEC/ASVS/MASTG IDs) into ranges for the printed cards, e.g. `[25, 26, 27] -> "25-27"`. It does this using `itertools.groupby`, which only detects consecutive runs correctly if the input is already sorted ascending with no duplicates:
```python
data_numbers = [int(s) for s in data] # no sort, no dedupe for k, g in groupby(enumerate(data_numbers), lambda x: x[0] - x[1]): ...
```
Several `source/*mappings*.yaml` files list numbers out of order or with duplicates (they're hand-maintained), which produces incorrect, non-ascending, or duplicated text on the generated cards.
### To Reproduce
1. `source/mobileapp-mappings-2.0.yaml`, suit `CRM`, card `CRMX`, tag `capec: [20, 116, 117, 97, 112, 485]`
2. Run this through the converter's `check_make_list_into_text()` (used by `build_template_dict()` for every card tag)
3. Output is `"20, 116-117, 97, 112, 485"` — not ascending, so the reader can't tell at a glance that `112` and `485` haven't already been covered.
A worse case, `source/webapp-mappings-2.2.yaml` suit `WC`, card `JOB`: `capec: [184, 242, 416, 438, 441, 444, 523, 518, 519, 548, 636, 691]` renders as `"184, 242, 416, 438, 441, 444, 523, 518-519, 548, 636, 691"` — `518-519` appears *after* `523`, which reads as a typo/data error to anyone using the card for actual ASVS/CAPEC lookups.
A duplicate case: `["5", "5", "6"]` renders as `"5, 5-6"` instead of `"5-6"`.
### Expected behavior
Reference numbers should render sorted ascending, de-duplicated, with consecutive runs grouped into ranges, regardless of the order they appear in the source YAML.
### Affected files (confirmed by scanning all mapping files)
`mobileapp-mappings-1.0.yaml`, `mobileapp-mappings-1.1.yaml`, `mobileapp-mappings-2.0.yaml`, `companion-mappings-1.0.yaml`, `webapp-mappings-2.2.yaml`, `webapp-mappings-3.0.yaml` — dozens of individual card entries.
### Proposed fix
Sort and de-duplicate before grouping:
```python
data_numbers = sorted(set(int(s) for s in data))
```
Existing unit tests only ever exercise pre-sorted, de-duplicated input, which is why this hasn't been caught. Happy to submit a PR with the fix plus regression tests using the real unsorted data above.
Contributor guide
Research direction
Start in scripts/convert.py at group_number_ranges() and trace its use through check_make_list_into_text(), which is called by build_template_dict(). Run the existing unit tests, then add regression coverage using the unsorted and duplicate reference lists; done means references render ascending, de-duplicated, with consecutive runs grouped into ranges.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- build-system, tooling
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 90/100