jf audit fail to scan pip dependencies

Describe the bug

if repo contain Python code with pip dependencies, xRay CLI failed to build dependency tree.

Current behavior

$ jf audit

16:18:26 [Debug] JFrog CLI version: 2.70.0
16:18:26 [Debug] OS/Arch: linux/amd64     
16:18:26 [Debug] Trace ID for JFrog Platform logs: dcba57f925c057e5
16:18:26 [Debug] Sending HTTP GET request to: https://<jfrog-server>/xsc/api/v1/system/version
16:18:26 [🔵Info] Log path: /home/<user>/.jfrog/logs/jfrog-cli.2024-09-30.16-18-26.2746758.log
💬The full scan results are available here: /tmp/jfrog.cli.temp.-1727708651-272837479

Vulnerable Dependencies                        
┌─────────────────────────────────────────────┐
│ ✨ No vulnerable dependencies were found ✨ │  
└─────────────────────────────────────────────┘
                                               
Secret Detection                               
┌──────────┬───────────────────┬─────────────┬─────────────────┐
│ SEVERITY │ FILE              │ LINE:COLUMN │ SECRET          │
├──────────┼───────────────────┼─────────────┼─────────────────┤
│ 🎃Medium │ ******** │ ***      │ ************ │
└──────────┴───────────────────┴─────────────┴─────────────────┘

Infrastructure as Code Vulnerabilities
┌────────────────────────────────────────────────────────────┐
│ ✨ No Infrastructure as Code vulnerabilities were found ✨ │
└────────────────────────────────────────────────────────────┘

Static Application Security Testing (SAST)
┌─────────────────────────────────────────────────────────────────────────┐
│ ✨ No Static Application Security Testing vulnerabilities were found ✨ │
└─────────────────────────────────────────────────────────────────────────┘
17:04:11 [Debug] Sending an error report to JFrog analytics...
17:04:11 [Debug] Sending HTTP GET request to: https://<jfrog-server>/xsc/api/v1/system/version
17:04:11 [Debug] failed to check availability of Xsc service:server response: 404 Not Found
404 page not found

Reporting to JFrog analytics is skipped...
17:04:11 [🔵Info] Trace ID for JFrog Platform logs: dcba57f925c057e5
17:04:11 [🚨Error] audit command in '/opt/repos/<repo-name>' failed:
[Thread 0]  Xray dependency tree scan request on 'pip' failed:
scanning pip dependencies failed with error: Get Dependencies Scan results...  executor timeout after 540 attempts with 5000 milliseconds wait intervals

$ cat /home//.jfrog/logs/jfrog-cli.2024-09-30.16-18-26.2746758.log

[Debug] Analytics metrics are disabled, skipping sending event request to XSC
[Debug] Sending HTTP GET request to: https://<jfrog-server>/xray/api/v1/system/version
[Debug] Usage Report: Sending info...
[Debug] Sending HTTP GET request to: https://<jfrog-server>/artifactory/api/system/version
[Debug] Sending HTTP GET request to: https://<jfrog-server>/xray/api/v1/entitlements/feature/contextual_analysis
[Debug] Artifactory response: 200
[Debug] JFrog Artifactory version is: 7.90.9
[Debug] Sending HTTP POST request to: https://<jfrog-server>/artifactory/api/system/usage
[Debug] Sending HTTP GET request to: https://<jfrog-server>/xray/api/v1/configuration/jas
[Debug] The path '/opt/repos/<repo-name>/.git/COMMIT_EDITMSG' is excluded
[Debug] The path '/opt/repos/<repo-name>/.git/FETCH_HEAD' is excluded
...
[Debug] The path '/opt/repos/<repo-name>/.gitignore' is excluded
[Debug] The path '/opt/repos/<repo-name>/.gitlab-ci.yml' is excluded
[Debug] The path '/opt/repos/<repo-name>/pytest.ini' is excluded
[Debug] The path '/opt/repos/<repo-name>/tests/test_func.py' is excluded
[Debug] mapped 1 working directories with indicators/descriptors:
{
  "/opt/repos/<repo-name>": [
    "/opt/repos/<repo-name>/requirements.txt"
  ]
}
[Debug] Detected 1 technologies at /opt/repos/<repo-name>: [pip].
[Info] Preforming 1 SCA scans:
[
  {
    "Target": "/opt/repos/<repo-name>",
    "Technology": "pip",
    "Descriptors": [
      "/opt/repos/<repo-name>/requirements.txt"
    ]
  }
]
[Debug] No pip.yaml configuration file was found. Resolving dependencies from pip default registry
[Info] Calculating Pip dependencies...
[Debug] Running /usr/bin/python3 -m venv venvdir
[Debug] Used "/usr/bin/python3" version: Python 3.12.4
[Debug] Failed running python venv: "/usr/bin/python3 -m venv venvdir" command failed: exit status 1 - Error: Command '['/tmp/jfrog.cli.temp.-1727705906-972550765/venvdir/bin/python3', '-m', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero exit status 1.

[Debug] Running virtualenv -p /usr/bin/python3 venvdir
[Debug] Running python -m pip install .
[Debug] Used "python" version: Python 3.12.4
[Debug] Running python -m pip install -r requirements.txt
[Debug] Created 'Pip' dependency tree with 19 nodes. Elapsed time: 9.3 seconds.
[Debug] Unique dependencies list:
[
    "pypi://oauthlib:3.2.2",
    "pypi://pyyaml:6.0.2",
    "pypi://urllib3:2.2.3",
    "pypi://charset-normalizer:3.3.2",
    "pypi://websocket-client:1.8.0",
    "pypi://pip:24.2",
    "pypi://certifi:2024.8.30",
    "pypi://durationpy:0.7",
    "pypi://cachetools:5.5.0",
    "pypi://pyasn1-modules:0.4.1",
    "pypi://rsa:4.9",
    "pypi://kubernetes:31.0.0",
    "pypi://pyasn1:0.6.1",
    "pypi://python-dateutil:2.9.0.post0",
    "pypi://six:1.16.0",
    "pypi://google-auth:2.35.0",
    "pypi://requests:2.32.3",
    "pypi://idna:3.10",
    "pypi://requests-oauthlib:2.0.0"
  ]
[Info] [Thread 0] Running SCA scan for /opt/repos/<repo-name> vulnerable dependencies in /opt/repos/<repo-name> directory...
[Info] Scanning 19 pip dependencies...
[Debug] JFROG_CLI_RELEASES_REPO is not set
[Debug] 'JFROG_CLI_RELEASES_REPO' environment variable is not configured. The Analyzer Manager app will be downloaded directly from releases.jfrog.io if needed.
[Debug] Sending HTTP POST request to: https://<jfrog-server>/xray/api/v1/scan/graph?scan_type=dependency
[Debug] Sending HTTP HEAD request to: https://releases.jfrog.io/artifactory/xsc-gen-exe-analyzer-manager-local/v1/1.9.3/linux-amd64/analyzerManager.zip
[Info] Waiting for scan to complete on JFrog Xray...
[Debug] Sending HTTP GET request to: https://<jfrog-server>/xray/api/v1/scan/graph/d0c6dfb3-1feb-4899-50f8-4492df094287?include_vulnerabilities=true
[Debug] Get Dependencies Scan results... (Attempt 1)
[Debug] Artifactory response: 200
[Info] [Thread 2] Running IaC scan...
[Info] [Thread 1] Running secrets scan...
[Debug] Secrets scanner input YAML:
scans:
    - roots:
        - /opt/repos/<repo-name>
      output: /tmp/jfrog.cli.temp.-1727705906-2351225783/Secrets_1727705916/results.sarif
      type: secrets-scan
      skipped-folders:
        - '**/*.git*/**'
        - '**/*node_modules*/**'
        - '**/*target*/**'
        - '**/*venv*/**'
        - '**/*test*/**'

[Debug] IaC scanner input YAML:
scans:
    - roots:
        - /opt/repos/<repo-name>
      output: /tmp/jfrog.cli.temp.-1727705906-2351225783/IaC_1727705916/results.sarif
      type: iac-scan-modules
      skipped-folders:
        - '**/*.git*/**'
        - '**/*node_modules*/**'
        - '**/*target*/**'
        - '**/*venv*/**'
        - '**/*test*/**'

[Debug] Executing /home/<user>/.jfrog/dependencies/analyzerManager/analyzerManager sec /tmp/jfrog.cli.temp.-1727705906-2351225783/Secrets_1727705916/config.yaml
[Debug] Executing /home/<user>/.jfrog/dependencies/analyzerManager/analyzerManager iac /tmp/jfrog.cli.temp.-1727705906-2351225783/IaC_1727705916/config.yaml
[Debug] Sending HTTP GET request to: https://<jfrog-server>/xray/api/v1/scan/graph/d0c6dfb3-1feb-4899-50f8-4492df094287?include_vulnerabilities=true
[Debug] Get Dependencies Scan results... (Attempt 2)
[Info] [Thread 2] Found 0 IaC vulnerabilities
[Info] [Thread 2] Running SAST scan...
[Debug] Sast scanner input YAML:
scans:
    - roots:
        - /opt/repos/<repo-name>
      type: sast
      exclude_patterns:
        - '**/*.git*/**'
        - '**/*node_modules*/**'
        - '**/*target*/**'
        - '**/*venv*/**'
        - '**/*test*/**'

[Debug] Executing /home/<user>/.jfrog/dependencies/analyzerManager/analyzerManager zd /tmp/jfrog.cli.temp.-1727705906-2351225783/Sast_1727705925/config.yaml /tmp/jfrog.cli.temp.-1727705906-2351225783/Sast_1727705925/results.sarif
[Debug] Sending HTTP GET request to: https://<jfrog-server>/xray/api/v1/scan/graph/d0c6dfb3-1feb-4899-50f8-4492df094287?include_vulnerabilities=true
[Debug] Get Dependencies Scan results... (Attempt 3)
[Info] [Thread 1] Found 1 secrets vulnerabilities
[Debug] Sending HTTP GET request to: https://<jfrog-server>/xray/api/v1/scan/graph/d0c6dfb3-1feb-4899-50f8-4492df094287?include_vulnerabilities=true
[Debug] Get Dependencies Scan results... (Attempt 4)
[Info] [Thread 2] Found 0 SAST vulnerabilities
[Debug] Sending HTTP GET request to: https://<jfrog-server>/xray/api/v1/scan/graph/d0c6dfb3-1feb-4899-50f8-4492df094287?include_vulnerabilities=true
[Debug] Get Dependencies Scan results... (Attempt 5)
[Debug] Sending HTTP GET request to: https://<jfrog-server>/xray/api/v1/scan/graph/d0c6dfb3-1feb-4899-50f8-4492df094287?include_vulnerabilities=true
[Debug] Get Dependencies Scan results... (Attempt 6)
[Debug] Sending HTTP GET request to: https://<jfrog-server>/xray/api/v1/scan/graph/d0c6dfb3-1feb-4899-50f8-4492df094287?include_vulnerabilities=true
[Debug] Get Dependencies Scan results... (Attempt 7)
[Debug] Sending HTTP GET request to: https://<jfrog-server>/xray/api/v1/scan/graph/d0c6dfb3-1feb-4899-50f8-4492df094287?include_vulnerabilities=true
[Debug] Get Dependencies Scan results... (Attempt 8)
[Debug] Sending HTTP GET request to: https://<jfrog-server>/xray/api/v1/scan/graph/d0c6dfb3-1feb-4899-50f8-4492df094287?include_vulnerabilities=true
[Debug] Get Dependencies Scan results... (Attempt 9)
......
[Debug] Sending HTTP GET request to: https://<jfrog-server>/xray/api/v1/scan/graph/d0c6dfb3-1feb-4899-50f8-4492df094287?include_vulnerabilities=true
[Debug] Get Dependencies Scan results... (Attempt 540)
[Debug] Sending HTTP GET request to: https://<jfrog-server>/xray/api/v1/scan/graph/d0c6dfb3-1feb-4899-50f8-4492df094287?include_vulnerabilities=true
[Debug] Get Dependencies Scan results... (Attempt 541)
[Debug] [Thread 1]  We couldn't find any vulnerable dependencies. Skipping....
[Debug] Analytics metrics are disabled, skipping sending update event request to XSC

Reproduction steps

jf audit in repo with Python code, or run Frogbot in CI-pipeline

Expected behavior

No response

JFrog CLI-Security version

JFrog CLI version: 2.70.0, Frogbot version: 2.21.13

JFrog CLI version (if applicable)

2.70.0

Operating system type and version

ubuntu 24.04

JFrog Xray version

No response

Start with the jf audit pip dependency path and reproduce the failure using a repository with requirements.txt or through Frogbot. Review the logged virtual-environment setup and dependency scan polling, then verify that the audit completes and reports the resolved Python dependencies instead of timing out after repeated scan attempts.