psf / psf/cachecontrol

testing with betamax?

Open
#212 5 comments 3 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
505
Forks
149
Avg merge
2d 13h
Merged PRs (30d)
7

Description

is there a way to test this module alongside betamax?

the idea would be that the cachecontrol module would hit its own cache if present, otherwise bounce back into betamax. I am already using betamax extensively in my project to "mock" HTTP requests (record and replay, really) and it's been quite useful. But when I pair it with cachecontrol, I don't get good results.

either:

  1. i add cachecontrol as an "old_adapter" in the betamax session, which works in the sense that requests never hit the network, but then they never hit the cachecontrol adapter either, because the query gets cached in betamax. even if they would hit cachecontrol, they would need to hit the network first
  2. I mount cachecontrol on top of a betamax session, in which case cachecontrol completely obliterates the betamax adapter and sends requests on the network (instead of tapping in betamax)

Those two possible scenarios are pretty much covered in this ifcase:

cache_adapter = cachecontrol.CacheControlAdapter(cache=FeedContentCacheStorage(self.db_path))
# assume we mount over http and https all at once so check
# only the latter
adapter = self._session.adapters.get('https://', None)
if hasattr(adapter, 'old_adapters'):
    # looks like a betamax session was setup, hook ourselves behind it
    logging.debug('appending cache adapter (%r) to existing betamax adapter (%r)', cache_adapter, adapter)
    adapter.old_adapters['http://'] = cache_adapter
    adapter.old_adapters['https://'] = cache_adapter
else:
    logging.debug('mounting cache adapter (%r)', cache_adapter)
    # override existing adapters to use the cache adapter instead
    self._session.mount('http://', cache_adapter)
    self._session.mount('https://', cache_adapter)

That's pretty much case 1 above, which falls back to case 2 if we don't detect a "betamax" adapter in place.

And of course, it doesn't work.

What I would need is something like betamax's "old_adapters" system. Instead of calling the parent class, CacheControlAdapter.send() should allow the caller to define a list of adapters to call instead. The way betamax does this is fairly simple: there's an old_adapters parameter that you can pass and that it uses for sending its requests..

I have looked into the cachecontrol test suite to figure out if there could be inspiration there, but it seems to "mock" an http adapter instead of delegating requests... Since I don't want to reimplement the entire thing myself (and I can't figure out how to hook betamax in there either), could we add a similar functionality here?

Thanks!

PS: it seems I can mock a parent class but that seems horribly dirty - I am not sure I want to go that far...

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with CacheControlAdapter.send(), the existing cachecontrol test suite, and Betamax's old_adapters handling. Define tests for cache hits and misses when CacheControl is combined with Betamax, with completion shown by requests being served from the cache when present and delegated without unintended network access otherwise.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
networking
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.