Consider increasing the default for num_sql_threads (currently 3)
Open
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 11.5k
- Forks
- 904
- Avg merge
- 4d 17h
- Merged PRs (30d)
- 18
Description
I ran a very rough micro-benchmark on the new num_sql_threads config option (added in #285)
datasette --config num_sql_threads:1 fivethirtyeight.db
Then
ab -n 100 -c 10 'http://127.0.0.1:8011/fivethirtyeight-2628db9/twitter-ratio%2Fsenators'
| Number of threads | Requests/second |
|---|---|
| 1 | 4.57 |
| 3 | 9.77 |
| 10 | 13.53 |
| 20 | 15.24 |
| 50 | 8.21 |
This was on my early 2018 OS X laptop. Need to benchmark in other common environments before making a decision on changing the default. That said, the default of 3 was a number I plucked out of thin air.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The issue points to the num_sql_threads config option added in #285; locate its implementation and current default first. Reproduce the supplied ab benchmark, compare thread counts in other common environments, and consider the default resolved when the results support a chosen value.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, sqlite
- Domain
- backend, databases, performance
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100