MemberJunction / MemberJunction/MJ

Semantic action search for agents with large action sets (>25 actions)

Open
#2,613 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
TSQL
Stars
29
Forks
6
Avg merge
2d 1h
Merged PRs (30d)
323

Description

> Backlog item carried over from closed draft PR [#2470](https://github.com/MemberJunction/MJ/pull/2470) (branch: `feature/agent-action-semantic-search`). A working in-progress prototype exists on that branch and can be revived when this is picked up.

## Motivation

When a `BaseAgent` is configured with many actions, the system prompt dumps the full list of action definitions every turn. This is the single largest contributor to system-prompt bloat for action-heavy agents:

- It scales linearly with the number of granted actions, even though only a handful are typically relevant to any given user turn.
- It crowds out the context budget that should be available for conversation history, working memory, and the actual user task.
- It increases cost and latency on every step of every run, not just the steps where actions are actually being chosen.

For agents with 50+ actions, the action dump alone can be larger than the rest of the system prompt combined. The cost is paid even when the model never invokes an action that turn.

## The approach (already prototyped on the closed PR branch)

Above a configurable **25-action threshold**, `BaseAgent` replaces the full action dump in the system prompt with:

1. A **category summary** of the agent's effective actions (what families of capability exist), and
2. Instructions for a built-in `_searchActions` meta-tool that the LLM can call like any other action.

The flow then becomes:

- LLM sees a compact category summary instead of N full action specs.
- When the model needs an action, it invokes `_searchActions(query, topK)` like any other action.
- The handler resolves via `AIEngine.FindSimilarActions` (existing embedding-based service — **no new infrastructure required**), ranks matches, and **scopes results to the agent's effective actions** (no leakage of other agents' actions).
- Results are returned as a standard `ActionResultSummary` so they flow through the normal action-results message path — downstream consumers don't need to change.
- **Below the threshold, behavior is unchanged.** Existing agents with few actions see no difference.

## Why this design (vs. alternatives)

- **Reuses existing infrastructure** — `AIEngine.FindSimilarActions` and the action embedding pipeline already exist; this just wires them into agent action selection.
- **No new message contract** — `_searchActions` returns the same `ActionResultSummary` everything else does, so the prompt path and audit trail don't fork.
- **Opt-in via threshold** — agents with small action sets pay zero cost. The threshold is configurable in `MJServer/config.ts` / `ServerBootstrap`.
- **Scoped retrieval** — the search is bounded to the agent's effective actions, preserving permission boundaries instead of just topK-over-all-actions.

## Code prototype already on the closed PR branch

- `packages/AI/Agents/src/agent-action-handler.ts` (~756 lines — new file extracting the action-handling logic from `base-agent.ts`).
- `packages/AI/Agents/src/base-agent.ts` (43 added / 432 removed — slimmed down once action handling moved out).
- `packages/AI/Agents/src/__tests__/action-search.test.ts` (~355 lines — threshold behavior, query parsing, scoping, result shape).
- `packages/AI/Agents/src/index.ts` — new export.
- `packages/MJServer/src/config.ts` (33 added — threshold config).
- `packages/ServerBootstrap/src/index.ts` (32 added — wiring).
- `.changeset/semantic-action-search-large-action-sets.md`.

## Open items when picked back up

These were on the PR's test plan but not yet ticked off:

- [ ] Unit-test threshold behavior (≤25 actions: full dump; >25: summary + meta-tool injection)
- [ ] Verify `_searchActions` invocation parses `query`/`topK` correctly and returns ranked matches
- [ ] Confirm matches are scoped to the agent's effective actions (no leakage of other agents' actions)
- [ ] Smoke-test an agent with 50+ actions end-to-end and confirm token savings vs. baseline
- [ ] Verify result shape matches existing `ActionResultSummary` consumers in the prompt path

## Why closing the PR for now

The work was opened as a draft for early visibility and is being parked rather than completed in-flight. Filing this issue so the design, prototype, and threshold-based approach aren't lost when the PR closes — the branch is still intact and can be reopened as a focused PR when this gets picked back up.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.