CedarCopilot / CedarCopilot/cedar-OS

feature: Agent Retrieved Context

Open
#157 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
172
Forks
19
PR merge metrics
No merged PRs in 30d

Description

# Request

The ability to return data from a frontend tool using `useRegisterFrontendTool` or similar back to the Mastra Agent so that the agent can retrieve context on-demand.

In my view this would be akin to a "reverse mention" in the Cedar state system since mentions are mechanisms for the User to subscribe the discrete context that they need for the task.

On the other hand, the Agent can use a similar system to retrieve discrete context from the frontend when it needs it. For example chart data, `useRef` data or even HTML tags that should not be a constant subscription.

An Agent equipped with these capabilities can behave similar to a terminal or coding agent with access to the filesystem. Where instead of adding the entire codebase into context or vectorizing it, the agent instead calls tools to read the data it finds necessary.

## Use Case

Slow data and large data. Potentially so large it could pollute the context if always used and should only be available when needed by mention or agent request.

Slow data may come in the form of network calls to other systems or serial operations that can only be performed from an embedding frontend. If you subscribe to this data directly you need to pause the chat for long periods or risk race conditions between messages and async data.

In either scenario, it's best to let Users or Agents decide when this data is needed.

## User & Agent Flow

1. User asks Agent to filter dashboard and change parameters. Then inspect a few charts and answer a question
2. The Agent calls state setting agentic actions to filter and parameterize the view
3. The Agent then calls tools to inspect each chart to get data that is correctly filtered
4. The Agent gets the data back from each chart it inspected and generates an answer

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.