Testing Center results report the routed topic as a truncated name ("p") for topics with the `p_<plannerId>_` prefix that `sf agent create` itself generates
- Dominant language
- No language data
- Stars
- 571
- Forks
- 80
- Avg merge
- 2d 15h
- Merged PRs (30d)
- 3
Description
CLI Version:
@salesforce/cli/2.149.9
Architecture:
win32-arm64
Node Version:
node-v24.14.1
Plugin Version:
@oclif/plugin-autocomplete 3.2.56 (core)
@oclif/plugin-commands 4.1.63 (core)
@oclif/plugin-help 6.2.58 (core)
@oclif/plugin-not-found 3.2.93 (core)
@oclif/plugin-plugins 5.4.87 (core)
@oclif/plugin-search 1.2.54 (core)
@oclif/plugin-update 4.7.59 (core)
@oclif/plugin-version 2.2.57 (core)
@oclif/plugin-warn-if-update-available 3.1.73 (core)
@oclif/plugin-which 3.2.61 (core)
@salesforce/cli 2.149.9 (core)
agent 2.0.5 (user)
apex 4.1.0 (core)
api 2.0.9 (core)
auth 5.0.6 (core)
code-analyzer 5.12.0 (user)
data 5.1.5 (core)
deploy-retrieve 4.1.2 (core)
info 4.0.9 (core)
limits 4.0.3 (core)
marketplace 2.0.5 (core)
org 6.0.9 (core)
packaging 3.0.5 (core)
schema 4.0.5 (core)
settings 3.0.5 (core)
sobject 2.0.5 (core)
telemetry 4.0.5 (core)
templates 57.0.9 (core)
trust 4.0.9 (core)
user 5.0.1 (core)
OS and Version:
Windows_NT 10.0.26200
Shell:
powershell
### Summary
`sf agent create` names an agent's topics with an auto-generated prefix:
`p__` (e.g. `p_16xxV0000006xXX_Shipping_Details`).
When an `AiEvaluationDefinition` test then runs against that agent, the run
results report the routed topic as the literal string **`p`** — in both
`generatedData.topic` and the topic assertion's `actualValue`. The name
appears to be truncated at the planner-id segment somewhere in the results
pipeline. Consequence: **every topic assertion fails**, no matter what
expected value the test author supplies, because nothing can equal `p`.
Built-in topics without the prefix (e.g. `Off_Topic`) are reported
correctly, which localizes the problem to the prefixed names.
This is server-side, not CLI parsing: `@salesforce/agents`
(`lib/agentTester.js`) spreads the API's `generatedData` payload through
verbatim (only `invokedActions` is HTML-decoded), so the `p` arrives in the
API response.
### Steps to reproduce
1. `sf agent create --spec specs/agentSpec.yaml --name "Test Agent" ...`
(topics get `p__*` developerNames — the default behavior)
2. Create a Testing Center test whose cases set `expectedTopic` to a
topic's full developerName; deploy and activate the agent.
3. `sf agent test run --api-name -o --wait 15 --json`
### Actual result
Every case: `topic_assertion: FAILURE` with
`expected: p__`, `actual: p`. The run's
`generatedData.topic` is also `p`.
### Expected result
The routed topic is reported by its full developerName (or the docs state
which name form topic assertions compare against).
### Workaround
Redeploy the topics under clean developerNames without the auto-generated
prefix (and update the planner bundle references); full names then
round-trip correctly and topic assertions pass. But note the colliding
defaults: the naming convention `agent create` generates is the one the
results pipeline cannot report.
### Environment
- sf CLI:
- Org: Agentforce Developer Edition (`orgfarm-*.develop.my.salesforce.com`)
Contributor guide
Research direction
Start with lib/agentTester.js and verify how the API's generatedData.topic and topic assertion actualValue are passed through. Run the listed sf agent create and sf agent test run reproduction steps, then compare prefixed and clean topic names in the JSON output. Done means determining whether the truncation is addressable in this CLI or must be fixed in the server-side results pipeline.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript, node.js
- Domain
- api, cli, testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100