[BUG] kickoff_for_each clears latest replay records before returning
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 58.8k
- Forks
- 8.5k
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 109
Description
Description
The documented synchronous, non-streaming Crew.kickoff_for_each replay contract is not preserved at current main / CrewAI 1.15.6 (b3aaaab023a53a08db4d36c9430b3463f1efcc7d).
The replay guide says kickoff_for_each supports replay from the most recent crew run:
docs/edge/.../replay-tasks-from-latest-crew-kickoff.mdxdocs/v1.15.6/.../replay-tasks-from-latest-crew-kickoff.mdx
Each copied kickoff already resets the shared latest-output store before executing and then persists its records. Because synchronous kickoff_for_each runs the copies sequentially, the final copy leaves the valid latest run. The parent method then performs another self._task_output_handler.reset() immediately before returning, deleting those records.
A deterministic regression on unmodified source receives both expected batch outputs but observes [] in the replay store after return. An ordinary-kickoff replay control passes.
Steps to Reproduce
-
Pin the repository and create its Python 3.12 environment:
git clone https://github.com/crewAIInc/crewAI.git git -C crewAI checkout --detach b3aaaab023a53a08db4d36c9430b3463f1efcc7d cd crewAI uv sync --python 3.12 --frozen --group dev -
Copy
test_kickoff_for_each_replay.pyfrom the public evidence repository intolib/crewai/tests/. -
Run the regression on unmodified production code:
./.venv/Scripts/python.exe -m pytest -q -n 0 --tb=short --disable-warnings \ lib/crewai/tests/test_kickoff_for_each_replay.py::test_kickoff_for_each_persists_only_the_most_recent_run_for_replay -
Run the ordinary-kickoff control:
./.venv/Scripts/python.exe -m pytest -q -n 0 --tb=short --disable-warnings \ lib/crewai/tests/test_kickoff_for_each_replay.py::test_regular_kickoff_persists_and_replays_latest_run
Observed on unmodified source:
- regression: 1 failed;
len(stored)is0, expected1; - ordinary-kickoff control: 1 passed;
- combined module: 1 failed, 1 passed.
Expected behavior
After one synchronous kickoff_for_each call over inputs first and latest:
- both batch outputs should return;
- the replay store should contain exactly the final run with
{"topic": "latest"}; Crew.replay(final_task_id)without an input override should execute again using the final stored input.
Screenshots/Code snippets
Failing assertion on the unmodified pinned source:
assert len(stored) == 1
E assert 0 == 1
E + where 0 = len([])
The root-cause-aligned candidate removes only the post-loop reset:
if not self.stream:
self.usage_metrics = total_usage_metrics
- self._task_output_handler.reset()
return results
With that deletion, the strengthened test verifies:
- final stored inputs;
- a third
Task.execute_synccall during replay; - the final interpolated description and returned output;
- a truthy persisted replay marker.
Operating System
Windows 11
Python Version
3.12.10
crewAI Version
1.15.6
crewAI Tools Version
1.15.6 (installed by the locked workspace; not involved in this test)
Virtual Environment
Venv (uv sync --python 3.12 --frozen --group dev)
Evidence
Public evidence repository:
https://github.com/HarperZ9/crewai-kickoff-replay-receipt
Read-back verification at commit cfe410c8cdce4e999bfeb9938193a45bfc9f22c8:
- public bundle verifier:
MATCH, 362 checks; - canonical result SHA-256:
2c9a17d1c6f049cecb8ef10917d4369ff68e626011f8fe222d329407d21e63aa; - Project Telos action receipt SHA-256:
e5b000658a704f1e099296a3005ddc13a8f3e6f620554c3c23365a7e38062aec; - candidate regression/control/combined: 1/1, 1/1, 2/2;
- selected existing
kickoff_for_eachtests: 8/8; - selected existing replay tests: 9/9;
- Ruff, formatting,
git diff --check, and reverse patch checks: pass.
The test substitutes Task.execute_sync with deterministic TaskOutput generation, so it exercises Crew copying, interpolation, SQLite persistence, stored-input recovery, and replay without a model call. Repository network blocking remains active on the exercised Python paths, with exact loopback hosts allowed for Windows socket.socketpair.
Possible Solution
Remove the terminal task-output reset from synchronous kickoff_for_each and add the focused regression/control coverage in the evidence repository.
I would be glad to submit the one-line production patch and tests if maintainers want this behavior.
Additional context
- Scope is synchronous, non-streaming, one-task execution. Async, streaming, multi-task, and real-model behavior were not validated.
- The selected nearby suites passed; the full repository suite was not run.
- Removing the reset means
kickoff_for_each([])preserves the previous replay snapshot instead of clearing it. That empty-input contract should be an explicit maintainer decision. - RED chronology is recorded but not cryptographically time-sealed beside an unmodified-source hash.
- This issue was prepared with AI assistance under human authorization. Per
.github/CONTRIBUTING.md, it must carry thellm-generatedlabel; I am requesting/applying that label explicitly.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Read the synchronous kickoff_for_each implementation in lib/crewai/src/crewai/crew.py, especially the post-loop task-output reset near lines 1106-1110. Copy test_kickoff_for_each_replay.py into lib/crewai/tests/ and run the two focused pytest commands from the issue. Done means both batch outputs return and the final replay record remains available for Crew.replay without an input override.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, sqlite
- Domain
- backend, databases, testing-qa
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 87/100