luckyframework / luckyframework/avram
Query cache issues with mosquito (and other single fiber code)
- Dominant language
- Crystal
- Stars
- 183
- Forks
- 67
- PR merge metrics
- No merged PRs in 30d
Description
The query cache creates a new instance on each fiber. In the context of HTTP requests, that means each request gets its own query cache. However, [Mosquito](https://github.com/mosquito-cr/mosquito) runs all of the jobs in the same fiber. This may be the case for sidekiq too 🤷♂️ In any case, this means that if you're running jobs less than 1.minute apart, each job after the first will pull from cache which could have some weird side affects.
For now, provided that you're booting your workers in a separate binary, you can just disable query cache for that.
```crystal
require "./app"
# Disable query cache because all jobs run in the
# same fiber which means infinite cache
Avram.settings.query_cache_enabled = false
Mosquito::Runner.start
```
Or alternatively, you could temporarily disable cache on a specific worker:
```crystal
def perform
Avram.temp_config(query_cache_enabled: false) do
# do job
end
end
```
Something we need to consider long term is maybe a method that allows for a specific query to bypass cache. Maybe something that looks like...
```crystal
UserQuery.new.without_cache do |query|
query.email("some@email.com").unconfirmed(true)
end
```
which is nice, but to quote @wout
> That would be great for specific queries, but not so much with validations.
Having cache in jobs isn't necessarily bad... we just have to make some considerations for how this can be handled. Another option which will fix mosquito specifically is that "hooks" are being implemented soon. A "before hook" would allow for starting each job with fresh cache.
Contributor guide
Research direction
Start by reviewing Avram's query-cache behavior and the existing query_cache_enabled and Avram.temp_config APIs described here. Compare the Mosquito single-fiber worker case with the suggested without_cache query API and the possible before-hook approach. Done should mean a clear, supported way to prevent stale cache results for repeated jobs without unnecessarily disabling caching elsewhere.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- crystal
- Domain
- database
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100