DependencyTrack / DependencyTrack/dependency-track

Improve CVE parsing performance if database latency is present

Open
#2,895 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Java
Stars
4.2k
Forks
811
Avg merge
8h 39m
Merged PRs (30d)
237

Description

### Current Behavior

Dependency track does 4000 commits per seconds for an hour to parse the CVE database when running in AWS EC2 m5d.xlarge with a local PostgreSQL on SSD storage. When switching to RDS PostgreSQL for the reason of having proper database management, backups, etc. the performance drops significantly, the commit rate is around 1500 commits / s.

In general, an application loading data into an SQL server should not commit the transaction so frequently. Even for a single record multiple commits are issued in DataNucleus, one for each field update.

### Proposed Behavior

Batch inserts and updates, and do commit less frequently to improve performance over database links with more latency than a local database.

### Checklist

- [X] I have read and understand the [contributing guidelines](https://github.com/DependencyTrack/dependency-track/blob/master/CONTRIBUTING.md#filing-issues)
- [X] I have checked the [existing issues](https://github.com/DependencyTrack/dependency-track/issues) for whether this enhancement was already requested

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.