sqlalchemy / sqlalchemy/dogpile.cache
Concurrent usage of dbm causes cache to go stale
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 299
- Forks
- 50
- PR merge metrics
- No merged PRs in 30d
Description
Migrated issue, originally created by David Gardner ()
Ran into this issue in production where users were reporting items in the cache being an hour old for a cache region configured with a one minute expiration time. Problem first observed with version 0.5.7, and reproduced with version 0.6.1.
I was able to reproduce the issue in a simple test of a function that returns datetime.now() (dogpile-test.py), if I run about 10 concurrent instances of the script after about a minute or so they will start reporting stale data.
However I noticed as a work-around if I use the MutexLock class from the lock_factory documentation, I don't run into the problem:
http://dogpilecache.readthedocs.io/en/latest/api.html?highlight=dogpile.cache.dbm#dogpile.cache.backends.file.DBMBackend.params.lock_factory
Attachments: dogpile-test.py
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the attached dogpile-test.py reproduction and the DBMBackend lock_factory documentation, then compare concurrent DBM usage with the MutexLock workaround. Reproduce the stale values under concurrent access and identify the expected expiration behavior; done means the DBM cache no longer serves expired data in that scenario.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100