jfrog / jfrog/frogbot

scan-multiple-repositories always clones source repo on Azure DevOps

Open
#629 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Go
Stars
372
Forks
107
Avg merge
2d 20h
Merged PRs (30d)
5

Description

Describe the bug

The scan-multiple-repositories command always clones the repo containing the frogbot-config.yml file instead of the repos defined in it
My pipeline + frogbot config are in a repo called CockpIT_frogbot
My frogbot config points to two repos CockpIT-front and CockpIT-back
As shown in the log, the config is read, but for both scans, the repo CockpIT_frogbot is cloned instead

Current behavior
##[section]Starting: Download and Run Frogbot
==============================================================================
Task         : Command line
Description  : Run a command line script using Bash on Linux and macOS and cmd.exe on Windows
Version      : 2.178.0
Author       : Microsoft Corporation
Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/command-line
==============================================================================
Generating script.
========================== Starting Command Output ===========================
[command]/usr/bin/bash --noprofile --norc /opt/agt/_work/_temp/e71a6319-464e-4155-973a-d17de31b9031.sh
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100  2395  100  2395    0     0  16292      0 --:--:-- --:--:-- --:--:-- 16292
Downloading the latest version of Frogbot...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100 27.9M  100 27.9M    0     0  76.0M      0 --:--:-- --:--:-- --:--:-- 76.0M
Frogbot downloaded successfully!
17:08:38 [Info] Frogbot version: 2.19.8
17:08:38 [Debug] Reading config from file system. Looking for .frogbot/frogbot-config.yml
17:08:38 [Debug] frogbot-config.yml found in /opt/agt/_work/1/s/.frogbot/frogbot-config.yml
17:08:38 [Debug] The content of frogbot-config.yml that will be used is:
- params:
    git:
      repoName: CockpIT-front
      branches:
        - master

- params:
    git:
      repoName: CockpIT-back
      branches:
        - master
17:08:38 [Debug] Locking config file to run config AddOrEdit command.
17:08:38 [Debug] Sending HTTP HEAD request to: 'https://github.com/jfrog/frogbot'
17:08:38 [Debug] Creating lock in: /tmp/jfrog.cli.temp.-1706371718-4135036626/locks/config
17:08:38 [Warn] couldn't extract payload from Access Token.
The provided access token is not a valid JWT, probably a reference token.
Some package managers only support basic authentication which requires also a username.
If you plan to work with one of those package managers, please provide a username.
17:08:38 [Debug] Releasing lock: /tmp/jfrog.cli.temp.-1706371718-4135036626/locks/config/jfrog-cli.conf.lck.398.1706371718273425349
17:08:38 [Debug] Config AddOrEdit command completed successfully. config file is released.
17:08:38 [Debug] Usage Report: Sending info...
17:08:38 [Info] Running Frogbot "scan-multiple-repositories" command
17:08:38 [Debug] Sending HTTP GET request to: https://<artifactory-server-url>/xray/api/v1/system/version
17:08:38 [Debug] Sending HTTP POST request to: https://usage-ecosystem.jfrog.io/api/usage/report
17:08:38 [Debug] Sending HTTP GET request to: https://<artifactory-server-url>/artifactory/api/system/version
17:08:38 [Debug] Artifactory response: 200 
17:08:38 [Debug] JFrog Artifactory version is: 7.55.10
17:08:38 [Debug] Sending HTTP POST request to: https://<artifactory-server-url>/artifactory/api/system/usage
17:08:38 [Debug] JFrog Xray version is: 3.71.6
17:09:08 [Debug] Can't check access to 'https://github.com/jfrog/frogbot', error while sending request:
Head "https://github.com/jfrog/frogbot": dial tcp 140.82.121.4:443: i/o timeout
17:09:08 [Debug] Setting timeout for go-git to 120 seconds ...
17:09:08 [Debug] Created temp working directory: /tmp/jfrog.cli.temp.-1706371748-2178551145
17:09:08 [Debug] Running git clone https://<azure-devops-server-url>/Global/CockpIT/_git/CockpIT_frogbot (master branch)...
17:09:08 [Debug] Project cloned from https://<azure-devops-server-url>/Global/CockpIT/_git/CockpIT_frogbot to /tmp/jfrog.cli.temp.-1706371748-2178551145
17:09:08 [Debug] Sending HTTP GET request to: https://<artifactory-server-url>/xray/api/v1/system/version
17:09:08 [Debug] Sending HTTP GET request to: https://<artifactory-server-url>/xray/api/v1/entitlements/feature/contextual_analysis
17:09:08 [Debug] The path '/tmp/jfrog.cli.temp.-1706371748-2178551145/.git' is excluded
17:09:08 [Debug] The path '/tmp/jfrog.cli.temp.-1706371748-2178551145/.git/HEAD' is excluded
17:09:08 [Debug] The path '/tmp/jfrog.cli.temp.-1706371748-2178551145/.git/config' is excluded
17:09:08 [Debug] The path '/tmp/jfrog.cli.temp.-1706371748-2178551145/.git/index' is excluded
17:09:08 [Debug] The path '/tmp/jfrog.cli.temp.-1706371748-2178551145/.git/objects' is excluded
17:09:08 [Debug] The path '/tmp/jfrog.cli.temp.-1706371748-2178551145/.git/objects/info' is excluded
17:09:08 [Debug] The path '/tmp/jfrog.cli.temp.-1706371748-2178551145/.git/objects/pack' is excluded
17:09:08 [Debug] The path '/tmp/jfrog.cli.temp.-1706371748-2178551145/.git/objects/pack/pack-c71ee7558fb4c6f988daa2bd1351a05dff7c589f.idx' is excluded
17:09:08 [Debug] The path '/tmp/jfrog.cli.temp.-1706371748-2178551145/.git/objects/pack/pack-c71ee7558fb4c6f988daa2bd1351a05dff7c589f.pack' is excluded
17:09:08 [Debug] The path '/tmp/jfrog.cli.temp.-1706371748-2178551145/.git/refs' is excluded
17:09:08 [Debug] The path '/tmp/jfrog.cli.temp.-1706371748-2178551145/.git/refs/heads' is excluded
17:09:08 [Debug] The path '/tmp/jfrog.cli.temp.-1706371748-2178551145/.git/refs/heads/master' is excluded
17:09:08 [Debug] The path '/tmp/jfrog.cli.temp.-1706371748-2178551145/.git/refs/remotes' is excluded
17:09:08 [Debug] The path '/tmp/jfrog.cli.temp.-1706371748-2178551145/.git/refs/remotes/origin' is excluded
17:09:08 [Debug] The path '/tmp/jfrog.cli.temp.-1706371748-2178551145/.git/refs/remotes/origin/master' is excluded
17:09:08 [Debug] The path '/tmp/jfrog.cli.temp.-1706371748-2178551145/.git/refs/tags' is excluded
17:09:08 [Debug] The path '/tmp/jfrog.cli.temp.-1706371748-2178551145/.git/shallow' is excluded
17:09:08 [Info] Couldn't determine a package manager or build tool used by this project. Skipping the SCA scan...
17:09:08 [Info] Xray scan completed
17:09:08 [Info] Didn't find vulnerable dependencies with existing fix versions for CockpIT-front
17:09:08 [Debug] Setting timeout for go-git to 120 seconds ...
17:09:08 [Debug] Created temp working directory: /tmp/jfrog.cli.temp.-1706371748-1524852656
17:09:08 [Debug] Running git clone https://<azure-devops-server-url>/Global/CockpIT/_git/CockpIT_frogbot (master branch)...
17:09:08 [Debug] Project cloned from https://<azure-devops-server-url>/Global/CockpIT/_git/CockpIT_frogbot to /tmp/jfrog.cli.temp.-1706371748-1524852656
17:09:08 [Debug] Sending HTTP GET request to: https://<artifactory-server-url>/xray/api/v1/system/version
17:09:08 [Debug] Sending HTTP GET request to: https://<artifactory-server-url>/xray/api/v1/entitlements/feature/contextual_analysis
17:09:08 [Debug] The path '/tmp/jfrog.cli.temp.-1706371748-1524852656/.git' is excluded
17:09:08 [Debug] The path '/tmp/jfrog.cli.temp.-1706371748-1524852656/.git/HEAD' is excluded
17:09:08 [Debug] The path '/tmp/jfrog.cli.temp.-1706371748-1524852656/.git/config' is excluded
17:09:08 [Debug] The path '/tmp/jfrog.cli.temp.-1706371748-1524852656/.git/index' is excluded
17:09:08 [Debug] The path '/tmp/jfrog.cli.temp.-1706371748-1524852656/.git/objects' is excluded
17:09:08 [Debug] The path '/tmp/jfrog.cli.temp.-1706371748-1524852656/.git/objects/info' is excluded
17:09:08 [Debug] The path '/tmp/jfrog.cli.temp.-1706371748-1524852656/.git/objects/pack' is excluded
17:09:08 [Debug] The path '/tmp/jfrog.cli.temp.-1706371748-1524852656/.git/objects/pack/pack-c71ee7558fb4c6f988daa2bd1351a05dff7c589f.idx' is excluded
17:09:08 [Debug] The path '/tmp/jfrog.cli.temp.-1706371748-1524852656/.git/objects/pack/pack-c71ee7558fb4c6f988daa2bd1351a05dff7c589f.pack' is excluded
17:09:08 [Debug] The path '/tmp/jfrog.cli.temp.-1706371748-1524852656/.git/refs' is excluded
17:09:08 [Debug] The path '/tmp/jfrog.cli.temp.-1706371748-1524852656/.git/refs/heads' is excluded
17:09:08 [Debug] The path '/tmp/jfrog.cli.temp.-1706371748-1524852656/.git/refs/heads/master' is excluded
17:09:08 [Debug] The path '/tmp/jfrog.cli.temp.-1706371748-1524852656/.git/refs/remotes' is excluded
17:09:08 [Debug] The path '/tmp/jfrog.cli.temp.-1706371748-1524852656/.git/refs/remotes/origin' is excluded
17:09:08 [Debug] The path '/tmp/jfrog.cli.temp.-1706371748-1524852656/.git/refs/remotes/origin/master' is excluded
17:09:08 [Debug] The path '/tmp/jfrog.cli.temp.-1706371748-1524852656/.git/refs/tags' is excluded
17:09:08 [Debug] The path '/tmp/jfrog.cli.temp.-1706371748-1524852656/.git/shallow' is excluded
17:09:08 [Info] Couldn't determine a package manager or build tool used by this project. Skipping the SCA scan...
17:09:08 [Info] Xray scan completed
17:09:08 [Info] Didn't find vulnerable dependencies with existing fix versions for CockpIT-back
17:09:08 [Info] Frogbot "scan-multiple-repositories" command finished successfully
##[section]Finishing: Download and Run Frogbot
Reproduction steps

No response

Expected behavior

Each repo declared in the frogbot-config.yml file should be cloned and scan instead of the repo containing the pipeline and config.

JFrog Frogbot version

2.19.8

Package manager info

maven/npm but not relevant

Git provider

Azure DevOps

JFrog Frogbot configuration yaml file

frogbot.yml (pipeline)

pr: none
trigger: none

pool: Linux-Build

variables:
  JF_GIT_PROJECT: $(System.TeamProject)
  JF_GIT_REPO: $(Build.Repository.Name)
  JF_GIT_API_ENDPOINT: $(System.CollectionUri)
  JF_GIT_BASE_BRANCH: $(Build.SourceBranchName)
  JF_GIT_OWNER: $(System.TeamProject)
  JF_GIT_PROVIDER: 'azureRepos'

jobs:
  - job:
    displayName: "Frogbot Scan Repository and Fix"
    steps:
       - task: CmdLine@2
         displayName: 'Download and Run Frogbot'
         env:
            JF_URL: $(JF_URL)
            JF_ACCESS_TOKEN: $(JF_ACCESS_TOKEN)
            JF_GIT_TOKEN: $(System.AccessToken)
            JF_RELEASES_REPO: "frogbot-generic-external"
            JFROG_CLI_LOG_LEVEL: "DEBUG"
         inputs:
            script: |
               getFrogbotScriptPath=$(if [ -z "$JF_RELEASES_REPO" ]; then echo "https://releases.jfrog.io"; else echo "${JF_URL}/artifactory/${JF_RELEASES_REPO}"; fi)
               curl -fLg "$getFrogbotScriptPath/artifactory/frogbot/v2/[RELEASE]/getFrogbot.sh" --header "X-JFrog-Art-Api: $JF_ACCESS_TOKEN" | sh
               ./frogbot scan-multiple-repositories

frogbot-config.yml

- params:
    git:
      repoName: CockpIT-front
      branches:
        - master

- params:
    git:
      repoName: CockpIT-back
      branches:
        - master
Operating system type and version

RHEL 8

JFrog Xray version

3.71.6

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at the scan-multiple-repositories command and its Azure DevOps repository-cloning path. Compare the configured repoName values with the repository URL and branch passed to each clone, using the supplied logs as the failing case. Done means CockpIT-front and CockpIT-back are each cloned and scanned instead of CockpIT_frogbot.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure, go
Domain
cli, devops
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.