Checking for DB updates can severely increase execution time
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 12.9k
- Forks
- 883
- Avg merge
- 3d 6h
- Merged PRs (30d)
- 25
Description
What happened:
Running grype consistently took over 10 seconds, even when there was no new DB to fetch. The dynamic UI makes it obvious that all other steps (like image pulling and scanning) had completely long ago by the time the DB update check completed. This timing is confirmed by running grype -vv ... and noting the seconds in the log entries.
What you expected to happen:
Grype should feel fast. And it should feel even faster when it's not pulling any new vulnerability data.
How to reproduce it (as minimally and precisely as possible):
time grype alpine:latest
Anything else we need to know?:
Using curl to fetch the "listing file" manually showed that this can take an unexpectedly long amount of time - over 3 seconds. But it can also take 0.1 second, too. In all of these curl attempts, the response included the header cf-cache-status: HIT.
Environment:
- Output of
grype version:
Application: grype
Version: 0.7.0
BuildDate: 2021-01-28T14:03:23Z
GitCommit: 8344b8f0d3f61729cf0845c08b31f26103e21231
GitTreeState: clean
Platform: darwin/amd64
GoVersion: go1.14.14
Compiler: gc
Supported DB Schema: 1
- OS (e.g:
cat /etc/os-releaseor similar):
macOS 11.2.1
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the delay with time grype alpine:latest, then run grype -vv ... and compare the timestamps around the database update check. Use the listing-file request described in the issue to investigate the variable response time. Done means the no-update path no longer adds several seconds to execution.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- databases, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100