Race condition on filesystems with low-precision mtime
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 3.1k
- Forks
- 233
- Avg merge
- 1h 44m
- Merged PRs (30d)
- 1
Description
Following up on #575, I realized that there's a possible (although very unlikely) race condition on file-systems where precision is only in seconds. During the same second:
- A file is written.
- The file is cached [by khal].
- A file is re-written.
The cache will now keep the first version of the file, and will believe it's up-to-date.
As a possible solution, I'd like to propose:
- Measure the mtime precision for each calendar at startup (similar to this).
- If the FS is lo-precision set some special internal flag to
slow_fstoTrue. - When reading files from disk if
mtime == int(datetime.now().timestamp())(ie: the file was written during this second), set the mtime in the cache to one second in the past (edit justmtime - 1will do).
This will result in the cache for a file written during the second is was read always expiring. The frequency of this happening should be very very low, but we'll avoid the race condition outlined above.
To summarize, this would add very very few occasional re-read, to avoid keeping stale data in the cache.
Rather than an actual PR, I wanted to discuss this first, because:
- Will we care enough about this?
- Maybe somebody has a better idea?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the mtime-precision measurement referenced in tests/conftest.py and tracing how file mtimes are used by khal's cache. Reproduce a same-second write, cache, and rewrite sequence, then verify that the cache does not retain stale data; the issue's proposed behavior is a starting point rather than an agreed solution.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- cli
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100