Memoized CrewBase methods retain per-instance Agent/Task/Crew in global cache
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 58.8k
- Forks
- 8.5k
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 109
Description
Summary
When @agent, @task, and @crew are used on a CrewBase class, memoized method results are stored in a module-level cache. Repeatedly creating new crew instances (for example, per request) leaves each instance's generated Agent, Task, and Crew objects strongly referenced in CacheHandler._cache, so process memory grows with request count.
Affected versions
- Confirmed in
1.15.2 - Re-checked in
1.15.20(wheel/source) and still present
Evidence in current implementation
crewai.project.utilsdefines globalcache = CacheHandler()crewai.project.annotationswrapsagent,task,crewdecorator outputs withmemoize(...)CacheHandleruses an internal plain dict without TTL/size bounds and stores entries as strong references
Reproduction (minimal)
- Define a
CrewBaseclass with at least one@agent,@task, and@crewmethod. - In a loop, create many new crew instances and touch cached methods.
- Inspect
cache._cacheand object liveness aftergc.collect().
Expected: old instance results can be reclaimed when user code drops references.
Observed: results remain in cache even after instance refs are removed.
Why this breaks long-running services
Some services build crews per request for isolation. Under steady request flow this cache behavior causes global accumulation of per-instance runtime objects (including nested clients and state), and RSS continues to rise across completed requests.
Suggested fix direction
- Make memoization cache scoped to the
CrewBaseinstance rather than module scope, or - Add an explicit invalidation/lifecycle API for CrewBase-bound memoized values, and call it when crews are discarded, while preserving backward compatibility for existing singleton-style usage.
If this behavior is expected for static class-like usage, documenting it would still help; however this makes per-request crew construction unsafe for memory-bounded processes.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Read CacheHandler and the global cache in crewai.project.utils, then inspect how agent, task, and crew decorators in crewai.project.annotations apply memoize. Reproduce the instance-retention case with the issue's loop and gc.collect() steps. Done means CrewBase-bound results no longer accumulate strongly across discarded instances, while existing singleton-style usage remains compatible and regression coverage verifies the lifecycle.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100