[Feature]: Freeze/lock/cache outputs for arbitrary nodes?
- Dominant language
- TypeScript
- Stars
- 4.7k
- Forks
- 388
- PR merge metrics
- No merged PRs in 30d
Description
### Feature Request
Especially when parsing large documents that may not be stable enough for the caching in the LLM requests to stick, it would be nice to save the output from arbitrary nodes or subgraphs to save costs during development and testing.
For example--pinging a web API might return slightly different results causing the entire chain to be processed even if the inputs don't change.
I would imagine this working either:
- TouchDesigner style where you click the lock button and the output is set to the current state for all further requests
- The way the chat node currently works where if the input does not change, the output also does not change.
### Code of Conduct
- [X] I agree to follow this project's Code of Conduct
Contributor guide
Research direction
Start by tracing the existing chat-node behavior and caching around LLM requests, then identify how arbitrary nodes and subgraphs are represented. Compare the requested locked-output and stable-input behaviors and define tests for unchanged inputs and changing web API results. Done means the chosen behavior is specified and covered for node and subgraph outputs.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- ai
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100