EDCD / EDCD/EDMarketConnector

EDDN - make properly threaded

Open
#1,786 3 comments 0 reactions 0 assignees View on GitHub
EDDN enhancement
Dominant language
Python
Stars
1.3k
Forks
182
Avg merge
1m
Merged PRs (30d)
1

Description

During the recent "use an sqlite3 database as the queue, and store the whole, wire-ready, message" changes I decided against actually making the sending threaded.

But the reasons for that were:

1. Wanting to try send of a message immediately, so in the main thread.
2. sqlite3, as currently implemented, not at all liking accessing the database from more than one thread.
1. You can't use the same sqlite3 handles in different python threads.
2. Opening the same database in more than one thread quickly runs into "database is locked" issues.

Thus I ended up using a tkinter timer to do delayed sending/retries, rather than a threaded worker.

Proposed solution:

1. Do not try to send immediately on the main thread.
2. Instead have a threaded worker, with it (outside of initial schema setup during startup) be the only thing accessing the database.
3. When a new message is received, attempt sending it once immediately.

This will also allow for increasing the sending timeout from the current 10s, which *might* be part of why users can see a send fail without any explicit indication of a problem at the server end.

Also, wrt multi-opening the sqlite3 database... it's worth checking if we just need to explicitly enable 'WAL', because last I checked the extra two files that entails were plain not being created.

Contributor guide

Open the contributing guide

Research direction

Start by tracing the EDDN queue, SQLite schema setup, and current tkinter timer/retry path described in the issue. Confirm the worker is the only post-startup database accessor, that a new message gets one immediate send attempt, and that retries and the longer timeout behave correctly; also investigate whether enabling SQLite WAL is needed.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, sqlite
Domain
backend, databases, networking
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.