apache / apache/ponymail-foal

Multi-threading issues; inconsistent locking

Open
#209 1 comment 0 reactions 0 assignees View on GitHub
bug
Dominant language
Python
Stars
45
Forks
23
PR merge metrics
No merged PRs in 30d

Description

https://github.com/apache/incubator-ponymail-foal/blob/bb4a6642506f0bb2c5a2a67570433536c3b22f66/tools/import-mbox.py#L154

According to [1], list.pop() is thread-safe, so the lock should not be needed.
However the documentation does not say that len(list) is thread-safe.
If list.pop() does need to be protected by a lock, then surely [2] should be similarly protected?

The FAQ [1] does say that incrementing an integer is not atomic, so the various global counts might not be incremented properly.

[1] https://docs.python.org/3/faq/library.html#what-kinds-of-global-value-mutation-are-thread-safe
[2] https://github.com/apache/incubator-ponymail-foal/blob/bb4a6642506f0bb2c5a2a67570433536c3b22f66/tools/import-mbox.py#L131

Contributor guide

No contributing guide indexed for this repository

Research direction

Read tools/import-mbox.py around lines 131 and 154, then review the linked Python thread-safety FAQ. Trace the global counters and list operations across the worker threads to determine whether the accesses use a consistent locking policy. Done means the locking and counter behavior is documented by the relevant tests or verification, with no remaining ambiguity between the two cited locations.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.