AppThreat / AppThreat/vulnerability-db

Seeking clarity on VDB full image disk expansion and cache-os build

Open
#254 25 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
151
Forks
23
Avg merge
22m
Merged PRs (30d)
7

Description

### Context

We are using VDB indirectly via **[OWASP dep-scan](https://github.com/owasp-dep-scan/dep-scan)** (`ghcr.io/owasp-dep-scan/dep-scan:latest`), which bundles and invokes VDB internally. All commands below are run through the dep-scan Docker image.

### Environment
- OS: Ubuntu 26.04 LTS (WSL2 on Windows 11)
- Docker: 29.0.5.1
- Architecture: x86_64
- RAM: 6GB allocated
- CPUs: 4

---

### Question 1: `--download-full-image` Results in 55GB on Disk vs 7.7GB Documented

The README states the full image is **7.7 GB**, but after running:

```bash
docker run --rm \
-v /opt/vdb:/opt/vdb \
-e VDB_HOME=/opt/vdb \
ghcr.io/owasp-dep-scan/dep-scan:latest vdb --download-full-image
```

The resulting files on disk were:

```
ls -lah
total 55G
drwxr-xr-x 2 user user 4.0K May 19 16:28 .
drwxr-xr-x 4 root root 4.0K May 19 15:58 ..
-rw-r--r-- 1 user user 23G May 19 08:34 data.index.vdb6
-rw-r--r-- 1 user user 32G May 19 08:30 data.vdb6
-rw-r--r-- 1 user user 204 May 19 16:22 vdb.meta
```

Is this expected behavior? If so, it would be very helpful to document the actual on-disk footprint separately from the compressed download size so users can plan disk capacity accordingly.

### Question 2: `--cache-os` Runs for a Long Time With No Progress Visibility

As an alternative to avoid the large disk footprint, `--cache-os` was attempted with heavy filtering to reduce scope since we needed only Ubuntu:

```bash
docker run --rm \
--cpus="4" \
--memory="5g" \
-v /opt/vdb:/opt/vdb \
-e VDB_HOME=/opt/vdb \
-e VDB_TEMP_DIR=/opt/vdb/tmp \
-e NVD_START_YEAR=2024 \
-e GITHUB_PAGE_COUNT=1 \
-e NPM_PAGE_COUNT=1 \
-e OSV_EXCLUDE_MALWARE=true \
-e VDB_IGNORE_ALMALINUX=true \
-e VDB_IGNORE_AMAZON=true \
-e VDB_IGNORE_ARCH=true \
-e VDB_IGNORE_CHAINGUARD=true \
-e VDB_IGNORE_OPENSUSE=true \
-e VDB_IGNORE_SUSE=true \
-e VDB_IGNORE_DEBIAN=true \
-e VDB_IGNORE_PHOTON=true \
-e VDB_IGNORE_REDHAT=true \
-e VDB_IGNORE_ROCKY=true \
-e VDB_IGNORE_ALPINE=true \
-e VDB_IGNORE_WOLFI=true \
-e LOG_LEVEL=DEBUG\
ghcr.io/owasp-dep-scan/dep-scan:latest \
vdb --cache-os
```

After **1+ hour** the only log output visible was:

```
INFO [2026-05-19 16:29:25,248] Refreshing AquaSource
```

With no further updates. It is impossible to tell whether the process is:
- Still actively downloading
- Stuck on a specific source
- Making any progress at all

Would it be possible to add more verbose progress logging per source, for example:
```
INFO Fetching NVD 2024... (1500/4200 records)
INFO Fetching GitHub Advisory... done (320 records)
INFO Indexing OSV... 45%
```

This would make a significant difference in user confidence that the build is progressing normally.

---

Any guidance would be appreciated.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.