Comfy-Org / Comfy-Org/ComfyUI

Create caching option that organizes cache based on loading times of cached values

Open
#8,367 1 comment 0 reactions 0 assignees View on GitHub
Feature help wanted
Dominant language
Python
Stars
133k
Forks
15.7k
Avg merge
1d 10h
Merged PRs (30d)
153

Description

### Feature Idea

Create a new caching algorithm/class that

- scores each cached output on the basis of (1) time to load cold and (2) relative size of cached output compared with available system memory
- arranges cached values in min- or max-heap such that the item being popped from cache is always the highest scored item

The effect is that we are more likely to pop items from the cache that take a small amount of time to load relative to their size.

Then expose that option via command line option, to exist alongside current options (lru, classic, no caching)

### Existing Solutions

_No response_

### Other

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.