pimutils / pimutils/khal

Race condition on filesystems with low-precision mtime

Open
#576 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

type: discussion
Dominant language
Python
Stars
3.1k
Forks
233
Avg merge
1h 44m
Merged PRs (30d)
1

Description

Following up on #575, I realized that there's a possible (although very unlikely) race condition on file-systems where precision is only in seconds. During the same second:

  1. A file is written.
  2. The file is cached [by khal].
  3. A file is re-written.

The cache will now keep the first version of the file, and will believe it's up-to-date.

As a possible solution, I'd like to propose:

  • Measure the mtime precision for each calendar at startup (similar to this).
  • If the FS is lo-precision set some special internal flag to slow_fs to True.
  • When reading files from disk if mtime == int(datetime.now().timestamp()) (ie: the file was written during this second), set the mtime in the cache to one second in the past (edit just mtime - 1 will do).

This will result in the cache for a file written during the second is was read always expiring. The frequency of this happening should be very very low, but we'll avoid the race condition outlined above.

To summarize, this would add very very few occasional re-read, to avoid keeping stale data in the cache.

Rather than an actual PR, I wanted to discuss this first, because:

  1. Will we care enough about this?
  2. Maybe somebody has a better idea?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the mtime-precision measurement referenced in tests/conftest.py and tracing how file mtimes are used by khal's cache. Reproduce a same-second write, cache, and rewrite sequence, then verify that the cache does not retain stale data; the issue's proposed behavior is a starting point rather than an agreed solution.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.