GuardianErrorExitCodeException: checkov completed with an Error exit code: 1. An error has occurred running the Checkov tool.

Hello All!

When i use almost vanilla MicrosoftSecurityDevOps@1 template:

parameters:
  TemplatesRepoName: ''

stages:
- stage: Microsoft_Defender
  displayName: Microsoft Defender for Cloud DevOps security
  condition: always()
  pool:
    vmImage: windows-latest
  jobs:
  - job: Microsoft_Defender_Scan
    displayName: Scan
    steps:
    - checkout: ${{ parameters.TemplatesRepoName }}
    - checkout: self
          
    - task: MicrosoftSecurityDevOps@1
      displayName: Microsoft Security DevOps
      inputs:
        config: templates/configs/checkov.gdnconfig
        categories: IaC

Inside pipeline:

resources:
  repositories:
    - repository: templates
      type: git
      name: PROJ/templates
      ref: refs/heads/feature/microsoft-defender-config

trigger:
- main

pr:
- main

stages:
- template: templates/microsoft-security.yml@templates
  parameters:
    TemplatesRepoName: templates

in the end, i got an error:

    Tool run time: 11.1929338 seconds
    ------------------------------------------------------------------------------
    Checkov completed with exit code 1
##[error]Error running checkov job: 1 of 1
##[error]GuardianErrorExitCodeException: checkov completed with an Error exit code: 1. An error has occurred running the Checkov tool.
    ------------------------------------------------------------------------------
    
    Process:
      Convert:
        Converting any raw tool logs to Sarif format ...
        Completed converting raw tool logs to Sarif format.
      Import:
        No tool logs to process. 
    Break:
      Guardian is searching for results that meet the given criteria to break the build.
      Results Query Summary:
        Baselines: default
        Suppression Sets: default
        Policy: azuredevops
      Saved file /home/vsts/work/1/a/.gdn/msdo.sarif
      Found no breaking results.
      Active results: 0
      Skipped results: 0

```ps
    Tool run time: 11.1929338 seconds
    ------------------------------------------------------------------------------
    Checkov completed with exit code 1
##[error]Error running checkov job: 1 of 1
##[error]GuardianErrorExitCodeException: checkov completed with an Error exit code: 1. An error has occurred running the Checkov tool.
    ------------------------------------------------------------------------------
    
    Process:
      Convert:
        Converting any raw tool logs to Sarif format ...
        Completed converting raw tool logs to Sarif format.
      Import:
        No tool logs to process. 
    Break:
      Guardian is searching for results that meet the given criteria to break the build.
      Results Query Summary:
        Baselines: default
        Suppression Sets: default
        Policy: azuredevops
      Saved file /home/vsts/work/1/a/.gdn/msdo.sarif
      Found no breaking results.
      Active results: 0
      Skipped results: 0
        Baselined results: 0
        Suppressed results: 0
        Results excluded by tool filters: 0
        Results below minimum severity: 0
        Results classified as Pass: 0
        Results in flight: 0
##[error]Error running tool 1 of 1: checkov
##[error]Error running checkov job: 1 of 1
##[error]GuardianErrorExitCodeException: checkov completed with an Error exit code: 1. An error has occurred running the Checkov tool.
##[error]BreakException: Guardian detected one or more breaking results.

My config file is really basic:

{
    "tools": [
      {
        "tool": {
          "name": "Checkov",
          "version": "Latest"
        },
        "arguments": {
          "DownloadExternalModules": "false",
          "TargetDirectory": "$(Checkov.DefaultTargetDirectory)"
        }
      }
    ]
  }

Even when i set DownloadExternalModules to false, i got an error in cmd:

/home/vsts/work/_msdo/packages/nuget/Microsoft.Guardian.CheckovRedist_linux_amd64.3.2.144/tools/dist/checkov --download-external-modules false --directory ./ --output-file-path /home/vsts/work/1/s/.gdn/.r/checkov/001/checkov.sarif
##[error]2024-07-08 13:06:05,846 [MainThread  ] [WARNI]  Failed to download module git::https://ORD@dev.azure.com/ORD/PROK/_git/keyvault//src?ref=v0.3:None (for external modules, the --download-external-modules flag is required)

That error is hilarious since flag is set to false but leave that...

The problem that i have is that i have a lot of errors from that pipeline:

For better reference, output with env variable set to DEBUG:
https://gist.github.com/michasacuer/c0e7127bfe537f1a15e19db5fcd8fa81

And also, sarif file is empty. This is an output from my code:

{
  "$schema": "https://schemastore.azurewebsites.net/schemas/json/sarif-2.1.0-rtm.5.json",
  "version": "2.1.0",
  "runs": [],
  "properties": {
    "producer": "MicrosoftSecurityDevOps"
  }
}

And scans tab in devops is empty.

When i don't use config file i got an output from msdo.safir file and scans tab has entries:

But task looks like this:

    - task: MicrosoftSecurityDevOps@1
      displayName: Microsoft Security DevOps
      inputs:
        categories: IaC

And still i got this error:

And output:

/home/vsts/work/_msdo/packages/nuget/Microsoft.Guardian.CheckovRedist_linux_amd64.3.2.144/tools/dist/checkov --directory ./ --output sarif --soft-fail --output-file-path /home/vsts/work/1/s/.gdn/.r/checkov/001/checkov.sarif
##[error]2024-07-08 13:00:23,785 [MainThread  ] [WARNI]  Failed to download module git::https://ORG@dev.azure.com/BarentzDevOps/PROJ/_git/keyvault//src?ref=v0.3:None (for external modules, the --download-external-modules flag is required)

So, to sum up:

1. When i provide config scans outputs are not saved to file
2. Even with variable set to false checkov yells that download modules var is required

Why it fails? What i do wrong?

Reproduce the MicrosoftSecurityDevOps@1 task with and without the config file, using the provided Checkov command output as the starting point. Inspect the generated checkov.sarif and msdo.sarif files and compare the task's handling of the DownloadExternalModules argument. Done means the configured scan produces usable SARIF results and the external-module behavior is consistent with the supplied setting.