lablup / lablup/backend.ai

OSV-Scanner workflow fails on every push to main

Open
#11,379 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
670
Forks
183
Avg merge
15h 13m
Merged PRs (30d)
368

Description

## Summary

The `OSV-Scanner` workflow (`.github/workflows/osv-scanner.yml`) fails on every push to `main`. Latest run: [25033560618](https://github.com/lablup/backend.ai/actions/runs/25033560618). The visible error — `Path does not exist: results.sarif` — is a downstream symptom; the actual root causes are stacked across three independent layers.

## Investigation

### `scan-containers` matrix (8/8 jobs failing)

Three independent bugs stack up:

1. `docker/build-push-action` does not pass `PYTHON_VERSION` build-arg. 5 of 8 dockerfiles use `FROM python:${PYTHON_VERSION`} (`backend.ai-{client,storage-proxy,webserver,appproxy-coordinator,appproxy-worker}.dockerfile`). With the ARG empty, buildx errors out:
```
ERROR: failed to build: failed to solve: failed to parse stage name "python:": invalid reference format
```
1. **The OSV-Scanner CLI is invoked with v1 syntax against a v2 binary.** `osv-scanner.yml:101-103` passes `--docker IMAGE`. v2.0.1 (the pinned action image) removed `--docker`; image scanning is now `osv-scanner scan image --archive image.tar`. Visible for the 3 non-Python images that DO build (`krunner-extractor`, `socket-relay`, `linuxkit-nsenter`) — the scanner prints help and exits without writing SARIF.
1. **No** `--output=results.sarif` passed. Even if scanning succeeded, SARIF goes to stdout; the subsequent `upload-sarif` can't find the file. (`scan-scheduled` doesn't hit this because `osv-scanner-reusable.yml` auto-injects `--output`.)

### `scan-scheduled / osv-scan` job

Working as designed but red. SARIF uploads to the Security tab successfully; the job exits non-zero because `fail-on-vuln: true` is the reusable-workflow default and current dev deps contain known vulns:

- `python-dotenv 0.20.0` → [GHSA-mf9w-mj56-hr94](https://osv.dev/GHSA-mf9w-mj56-hr94) (`requirements.txt`)
- `black 24.8` → [GHSA-3936-cmfr-pm3m](https://osv.dev/GHSA-3936-cmfr-pm3m) (`tools/black-requirements.txt`)
- `pytest 8.3.3` → [GHSA-6w46-j5rx-g56g](https://osv.dev/GHSA-6w46-j5rx-g56g) (`tools/pytest-requirements.txt`)

### Bonus: `sbom.yml` has the same bug latent

`.github/workflows/sbom.yml:85-93` builds the same dockerfiles without passing `PYTHON_VERSION`. Silent today because the workflow is `workflow_dispatch`-only and hasn't been triggered since 2025-04-16, but it would fail identically the next time it's triggered.

## Plan

Tracked as sub-issues (linked below). Likely **2 PRs**: one folding sub-issues 1 + 3 (same root-cause fix in two workflow files), one for sub-issue 2 (dev-dep upgrades).

While in `osv-scanner.yml`, also bump `github/codeql-action/upload-sarif@v3` → `@v4` ([v3 deprecation Dec 2026](https://github.blog/changelog/2025-10-28-upcoming-deprecation-of-codeql-action-v3/)).

JIRA Issue: BA-5884

Contributor guide

Open the contributing guide

Research direction

Start with .github/workflows/osv-scanner.yml, focusing on the scan-containers matrix and the scanner invocation, then compare it with .github/workflows/osv-scanner-reusable.yml. Inspect .github/workflows/sbom.yml for the parallel build issue and the named requirements files for vulnerable dependencies. Done means the affected workflows build and scan successfully, produce results.sarif for upload, and handle the documented dependency findings.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, github-actions, python
Domain
ci-cd, devops, security
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.