EDDN - make properly threaded
- Dominant language
- Python
- Stars
- 1.3k
- Forks
- 182
- Avg merge
- 1m
- Merged PRs (30d)
- 1
Description
During the recent "use an sqlite3 database as the queue, and store the whole, wire-ready, message" changes I decided against actually making the sending threaded.
But the reasons for that were:
1. Wanting to try send of a message immediately, so in the main thread.
2. sqlite3, as currently implemented, not at all liking accessing the database from more than one thread.
1. You can't use the same sqlite3 handles in different python threads.
2. Opening the same database in more than one thread quickly runs into "database is locked" issues.
Thus I ended up using a tkinter timer to do delayed sending/retries, rather than a threaded worker.
Proposed solution:
1. Do not try to send immediately on the main thread.
2. Instead have a threaded worker, with it (outside of initial schema setup during startup) be the only thing accessing the database.
3. When a new message is received, attempt sending it once immediately.
This will also allow for increasing the sending timeout from the current 10s, which *might* be part of why users can see a send fail without any explicit indication of a problem at the server end.
Also, wrt multi-opening the sqlite3 database... it's worth checking if we just need to explicitly enable 'WAL', because last I checked the extra two files that entails were plain not being created.
Contributor guide
Research direction
Start by tracing the EDDN queue, SQLite schema setup, and current tkinter timer/retry path described in the issue. Confirm the worker is the only post-startup database accessor, that a new message gets one immediate send attempt, and that retries and the longer timeout behave correctly; also investigate whether enabling SQLite WAL is needed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, sqlite
- Domain
- backend, databases, networking
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100