Onboarding with Agent needs improvements
- Dominant language
- C#
- Stars
- 135
- Forks
- 260
- Avg merge
- 3d 1h
- Merged PRs (30d)
- 143
Description
### Scenario: Azure SDK Tools Agent Onboarding Experience Issues
I was updating our EngHub documentation to make the Azure SDK Tools Agent more prominent, specifically within the onboarding section. Our primary onboarding mechanism remains Release Planner, but to highlight the agent, I wanted to evaluate what the onboarding experience looks like when initiated via the agent.
From within VS Code, I entered the prompt:
> “Help me onboard my new product to the Azure SDK.”
The agent correctly validated that I had the required MCP server dependencies. However, once it responded, the experience diverged significantly from how onboarding actually works today.
#### 1. Incorrect Initial Guidance
The agent immediately presented a section titled **“Azure SDK onboarding steps”**, which included:
* Initializing a TypeSpec project
* Creating a service label
* Validating TypeSpec
* Creating a release plan
* Generating SDK code
This entire section is problematic and should not appear during onboarding.
Onboarding occurs during **private preview**, and at that stage:
* SDK generation is **not required**
* TypeSpec initialization may not exist yet
* Teams are not expected to validate specs or generate SDKs
These steps are post-onboarding activities and should not be implied as prerequisites. This creates confusion and sets incorrect expectations for service teams.
#### 2. Misaligned Question Set
After the initial steps, the agent asked the following questions:
1. What is your service or product name?
2. Is this a management plane (ARM) or data plane API?
3. Do you already have an existing Swagger or OpenAPI spec?
4. Which SDK languages do you need?
Only some of these questions align with actual onboarding requirements.
The fourth question is particularly concerning. We should **not** be asking teams which SDK languages they want. Tier-one languages are mandatory and predefined. In fact, I am not even sure language selection is part of the onboarding form at all.
This raises a broader issue: the agent appears to be inventing an onboarding flow rather than strictly reflecting the **actual onboarding form and process**.
#### 3. What the Onboarding Experience Should Do
When a user asks to onboard via the agent, the experience should be tightly scoped and form-driven.
Specifically:
* Only ask for information that is required to complete the **official onboarding form**
* Do not introduce SDK generation or TypeSpec setup concepts
* Treat onboarding as an administrative and discovery step, not an implementation workflow
At minimum, we need:
* Product or service identification
* Ideally via **Service Tree ID**
* If the user does not have it, we need a way to resolve the correct Service Tree entry from a product name
* Plane type, if required by the form
* Any other fields that are explicitly required today
Nothing more.
#### 4. Recommendation
The agent’s onboarding flow needs to be re-evaluated and constrained to mirror the real onboarding process exactly. Today, it is:
* Over-prescriptive
* Misaligned with private preview reality
* Asking questions that may not even exist in the onboarding form
* Introducing steps that occur much later in the lifecycle
The goal should be a **lightweight, accurate, and form-completion-oriented experience**, not a pseudo end-to-end SDK workflow.
Contributor guide
Assessment
This issue has not been assessed yet.