sqlalchemy / sqlalchemy/dogpile.cache
option to drop module/function from key_generator functions in util.py
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 299
- Forks
- 50
- PR merge metrics
- No merged PRs in 30d
Description
Migrated issue, originally created by jvanasco (jvanasco)
I tracked down a performance issue to usage of dogpile.cache.region:Region.cache_on_arguments in a few places.
The default key generation is provided key generator functions in dogpile.cache.util, with this relevant code shared by all:
if namespace is None:
namespace = '%s:%s' % (fn.__module__, fn.__name__)
else:
namespace = '%s:%s|%s' % (fn.__module__, fn.__name__, namespace)
My issue was the "base" of the key('%s:%s' % (fn.__module__, fn.__name__)) was incredibly long in a handful of the most-used places, and the keys themselves were taking up a significant chunk of the memory allocated to Redis. Dropping the key length reclaimed a lot of space.
While a key_mangler would solve this, I needed to keep the keys unmangled. The easiest solution was to just reimplement the stock key generator to just use the namespace argument as the key.
This seems like a relatively useful approach for many people, so I wanted to suggest porting it upstream:
I think the relevant changes would be:
- The key_generators have a new kwarg to omit the module+name prefix (like to_str)
- cache_on_arguments(_multi) have a new kwarg for the same, which is just passed to the key_generator (again, like to_str)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in util.py with the shared key generator functions, then trace cache_on_arguments and cache_on_arguments_multi to see how their keyword arguments reach the generators. Verify that the new option is accepted and forwarded consistently, and that generated keys can omit the module-and-function prefix while preserving the namespace behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, redis
- Domain
- api, backend
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100