aws-cloudformation / aws-cloudformation/cloudformation-guard

[Enhancement] Format junit reports for clarity

Open
#659 2 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Rust
Stars
1.4k
Forks
196
Avg merge
3d 6h
Merged PRs (30d)
5

Description

**Is your feature request related to a problem? Please describe.**

When using `cfn-guard validate` with `--output-format junit`, the generated JUnit XML reports contain non-compliant logs but lack line number and column information in the proper sections of the JUnit report structure. This makes it difficult for CI/CD tools and IDEs to provide precise navigation to the exact location of compliance violations in CloudFormation templates.

Currently, users need to implement custom post-processing logic to parse the failure messages and extract location information. For example, in the AWS CDK Security Guardian tool, we have to manually parse and reformat the XML output to make it more useful for developers (see: https://github.com/aws/aws-cdk/blob/main/tools/%40aws-cdk/security-guardian/src/cfn-guard-runner.ts).

**Describe the solution you'd like**

Enhance the JUnit XML output format to include line number and column information in standard JUnit XML attributes. Specifically:

1. Add `line` and `column` attributes to `` elements to indicate where the violation occurred
2. Include `file`, `line`, and `column` attributes in `` elements for precise error location
3. Ensure the location information is extracted from the Guard evaluation context and properly formatted

Example desired output:
```xml



Detailed failure message with context...

```

This would allow CI/CD systems, IDEs, and other tools to automatically navigate to the exact location of compliance violations without requiring custom parsing logic.

**Describe alternatives you've considered**

1. **Post-processing the JUnit XML** (current approach): Parse the XML output and extract location information from failure messages, then inject it into the proper XML attributes. This works but requires every user to implement their own parsing logic and is fragile if message formats change.

2. **Using structured output format**: The `--structured` flag provides JSON output with more detailed information, but this requires completely different tooling and doesn't integrate with standard JUnit report consumers.

3. **Parsing console output**: Extract location information from the human-readable console output, but this is even more fragile and doesn't work well with CI/CD pipelines.

**Additional context**

- The Guard evaluation engine already has access to line and column information during rule evaluation
- JUnit XML format supports custom attributes, so this enhancement would be backward compatible
- This would benefit all Guard users, not just those using it with AWS CDK
- Many static analysis tools (ESLint, cfn-lint, etc.) already provide location information in their JUnit outputs
- Reference implementation showing the need for post-processing: https://github.com/aws/aws-cdk/blob/main/tools/%40aws-cdk/security-guardian/src/cfn-guard-runner.ts

This enhancement would make CloudFormation Guard's JUnit output more useful out-of-the-box and align it with industry standards for static analysis tool reporting.

Contributor guide

Open the contributing guide

Research direction

Start with the cfn-guard validate command's --output-format junit path and trace how the Guard evaluation context reaches the JUnit XML formatter. Review the referenced AWS CDK security-guardian implementation for expected location handling. Done means testcase and failure elements include the available file, line, and column values while preserving existing JUnit output compatibility.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
cli, testing
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.