Docs: add OrcaReplay to the Observability section
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 58.8k
- Forks
- 8.5k
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 109
Description
This issue was authored by an AI agent (Claude Code), as required by CONTRIBUTING.md. I could not apply the
llm-generatedlabel myself — I do not have permission on this repository — so I am declaring it here and would be grateful if a maintainer adds it.
Feature Area
Documentation
Is your feature request related to an existing bug?
No. This is a documentation addition.
Describe the solution you'd like
Add a page for OrcaReplay under Observability, alongside the existing guides.
OrcaReplay is an open-source (Apache-2.0) record/replay debugger for agents. It records a run to a file, then replays that file with the model served from the recording — so a run reproduces exactly, offline, without spending tokens, and can be forked from any step onto a different model.
It differs from the tools currently in that section in one way worth stating: there is nothing to add to your crew. No SDK, no callback handler, no import.
orca record generic-openai -- python your_crew.py
orca replay last
The page would cover the command, what gets captured, replay and forking, and three CrewAI-specific notes that were measured against crewai 1.15.20 rather than assumed:
- 1.x moved LiteLLM to an optional extra, so a default install reaches OpenAI through
crewai.llms.providers.openai.completion, which reads bothOPENAI_API_BASEandOPENAI_BASE_URL - a bare model name always reaches that provider, while a prefixed one no native provider claims falls back to LiteLLM — so
LLM(model="openai/my-gateway-model")resolves only where LiteLLM is installed. Relevant when pointing a crew at a gateway - CrewAI's telemetry is a separate connection, and the page shows how to turn it off
CrewAI also has its own check in OrcaReplay's CI — a real Agent, Task and Crew recorded against a stub origin, the origin stopped, and the recording replayed offline — so the claim the page makes fails a build rather than ageing quietly.
Describe alternatives you've considered
Leaving it out. The reason not to is that CrewAI users currently have no documented way to reproduce a run offline or to compare two models on the same task, and the tools listed do not do that — they are trace destinations rather than replayers.
Additional context
I opened #7364 with the page before reading the contributing guide properly, and it was closed automatically for having no linked issue. My mistake — I checked the repository root for a contributing guide and missed that it lives under .github/. This issue is that prerequisite; I will open a fresh PR referencing it, in all four locales, and declare the AI authorship there too.
Willingness to Contribute
Yes, I can submit a PR
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Read .github/CONTRIBUTING.md first, then inspect the existing Observability guides and their four locale equivalents. Document OrcaReplay's record, replay, forking, CrewAI provider, telemetry, and CI-tested offline workflow, with the page added in all four locales.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- documentation
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100