galaxyproject / galaxyproject/galaxy
[RFC] Caching Framework
- Dominant language
- Python
- Stars
- 1.9k
- Forks
- 1.2k
- Avg merge
- 2d 16h
- Merged PRs (30d)
- 169
Description
Given recent PRs like https://github.com/galaxyproject/galaxy/pull/4495 and https://github.com/galaxyproject/galaxy/pull/4546 it seems like Galaxy might benefit from a caching framework.
I would propose:
- TTLCache, allowing things to expire even if we, in the code, fail to expire them when e.g. new tools are installed.
- A pluggable caching framework, the default being an in-memory cache object. I've used https://pypi.python.org/pypi/cachetools and been happy with it. I'll contribute a redis cache backend or something like this.
- We look at aggresively caching things that aren't changed often, e.g. the toolbox.
- We have cache keys including useful information, e.g. the toolbox might be under `tools.toolbox`, so we can decide when a new tool is installed, we should expire everything with `tools.` in it / prefixing it.
cc @mvdbeek how would you feel about this?
Contributor guide
Research direction
Review PRs 4495 and 4546 first, then compare the proposed TTLCache, pluggable backends, toolbox caching, and prefixed cache keys with cachetools. The issue names no implementation files or tests; done would require an agreed design and implementation plan before coding can begin.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, redis
- Domain
- backend, performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100