argoproj / argoproj/argo-workflows

3.5.8+: improve SQLite performance with thread pools

Open
#13,638 0 comments 0 reactions 0 assignees View on GitHub
area/api area/server type/regression
Dominant language
Go
Stars
17k
Forks
3.7k
Avg merge
1d 20h
Merged PRs (30d)
138

Description

### Pre-requisites

- [X] I have double-checked my configuration
- [X] I have tested with the `:latest` image tag (i.e. `quay.io/argoproj/workflow-controller:latest`) and can confirm the issue still exists on `:latest`. If not, I have explained why, **in detail**, in my description below.
- [X] I have searched existing issues and could not find a match for this bug
- [ ] I'd like to contribute the fix myself (see [contributing guide](https://github.com/argoproj/argo-workflows/blob/main/docs/CONTRIBUTING.md))

### What happened? What did you expect to happen?

Following up on https://github.com/argoproj/argo-workflows/pull/13166 / https://github.com/argoproj/argo-workflows/pull/13166#issuecomment-2167263924 -- thread pooling was never added, which would make the SQLite access more performant than a mutex

### Version(s)

v3.5.8 and above

### Paste a minimal workflow that reproduces the issue. We must be able to run the workflow; don't enter a workflows that uses private images.

```YAML
n/a, this is server specific
```

### Logs from the workflow controller

```text
n/a, this is server specific
```

### Logs from in your workflow's wait container

```text
n/a, this is server specific
```

Contributor guide

Open the contributing guide

Research direction

Read pull request #13166 and its linked discussion first, then trace the server-specific SQLite access described there. The work is done when SQLite access uses thread pooling as proposed and its performance can be compared with the existing mutex-based approach.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, sqlite
Domain
backend, databases, performance
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.