argoproj / argoproj/argo-workflows
3.5.8+: improve SQLite performance with thread pools
- Dominant language
- Go
- Stars
- 17k
- Forks
- 3.7k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 138
Description
### Pre-requisites
- [X] I have double-checked my configuration
- [X] I have tested with the `:latest` image tag (i.e. `quay.io/argoproj/workflow-controller:latest`) and can confirm the issue still exists on `:latest`. If not, I have explained why, **in detail**, in my description below.
- [X] I have searched existing issues and could not find a match for this bug
- [ ] I'd like to contribute the fix myself (see [contributing guide](https://github.com/argoproj/argo-workflows/blob/main/docs/CONTRIBUTING.md))
### What happened? What did you expect to happen?
Following up on https://github.com/argoproj/argo-workflows/pull/13166 / https://github.com/argoproj/argo-workflows/pull/13166#issuecomment-2167263924 -- thread pooling was never added, which would make the SQLite access more performant than a mutex
### Version(s)
v3.5.8 and above
### Paste a minimal workflow that reproduces the issue. We must be able to run the workflow; don't enter a workflows that uses private images.
```YAML
n/a, this is server specific
```
### Logs from the workflow controller
```text
n/a, this is server specific
```
### Logs from in your workflow's wait container
```text
n/a, this is server specific
```
Contributor guide
Research direction
Read pull request #13166 and its linked discussion first, then trace the server-specific SQLite access described there. The work is done when SQLite access uses thread pooling as proposed and its performance can be compared with the existing mutex-based approach.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, sqlite
- Domain
- backend, databases, performance
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100