Dynamically determine caching's time_threshold

But time thresholds should depend on the latency of the caching backend, shouldn't they? We definitely don't want to cache every function call to memcached and definitely not to disk, so that a 0 here is not good for the users.

The default 0.3 was a hacky way of avoiding that, agreed. Instead what we need is an estimate of the latency in the backend, and the time of computation for the wrapped function . If in the first call the latter takes longer than the latency by a factor of, say 2, then we store to cache and upon the next call, check these numbers again and retrieve from cache. Backends can define a default latency or, ideally, they can precompute it dynamically upon construction by averaging a few hundred dummy read/writes.

I think we should do this since it avoids a major potential performance issue and just takes a couple of calls to `time()`

_Originally posted by @mdbenito in https://github.com/aai-institute/pyDVL/pull/458#discussion_r1420281580_