Azure / Azure/azure-rest-api-specs

TypeSpec Validation Experience Improvements

Open
#35,833 1 comment 0 reactions 1 assignee Claimed by @mikeharder View on GitHub
Dominant language
TypeSpec
Stars
3.1k
Forks
5.9k
Avg merge
2d 22h
Merged PRs (30d)
444

Description

Opening an issue on items I think could be done to improve the TypeSpec Validation CI/local run

### CI only
- Buffer the output so the collpased group can directly show if it succeeded or failed

Image

Instead produce something like
- ✔️ Validating specification/compute/Compute.Management
- ❌ Validating specification/compute/Disk.Management
- ✔️ Validating specification/compute/Gallery.Management
- ✔️ Validating specification/compute/Sku.Management

- Run git diff with colors [Example](https://github.com/Azure/typespec-azure/blob/cfb35e54dd5ea9ea35e13ddc8bf2972ffbc1515b/.github/workflows/external-integration.yml#L81)

### Tool
- Summary of which rule failed at the end/less noise
- Related to that see if each rule can be run with `ora` or something like that that will keep it nice and grouped with status check
- For rules with sub rules like tspconfig validation would be nice to format those like typespec errors/warnings with colors

```
Validation on option "options.@azure-tools/typespec-csharp.package-dir" in "@azure-tools/typespec-csharp" are failed. However, per @azure-tools/typespec-csharp’s decision, we will treat it as passed.
Validation on option "options.@azure-tools/typespec-csharp.namespace" in "@azure-tools/typespec-csharp" are failed. However, per @azure-tools/typespec-csharp’s decision, we will treat it as passed.
Validation on option "options.@azure-tools/typespec-csharp.clear-output-folder" in "@azure-tools/typespec-csharp" are failed. However, per @azure-tools/typespec-csharp’s decision, we will treat it as passed.
Validation on option "options.@azure-tools/typespec-csharp.package-dir" in "@azure-tools/typespec-csharp" are failed. However, per @azure-tools/typespec-csharp’s decision, we will treat it as passed.
[SdkTspConfigValidation]: validation failed.
- Failed to find "options.@azure-tools/typespec-python.package-dir" with expected value "/^azure-mgmt(-[a-z]+){1,2}$/". Please add "options.@azure-tools/typespec-python.package-dir" with expected value "/^azure-mgmt(-[a-z]+){1,2}$/".
- Failed to find "options.@azure-tools/typespec-python.namespace" with expected value "/^azure\.mgmt(\.[a-z]+){1,2}$/". Please add "options.@azure-tools/typespec-python.namespace" with expected value "/^azure\.mgmt(\.[a-z]+){1,2}$/".
- Failed to find "options.@azure-tools/typespec-python.generate-sample" with expected value "true". Please add "options.@azure-tools/typespec-python.generate-sample" with expected value "true".
- Failed to find "options.@azure-tools/typespec-python.generate-test" with expected value "true". Please add "options.@azure-tools/typespec-python.generate-test" with expected value "true".
- Failed to find "options.@azure-tools/typespec-csharp.package-dir" with expected value "/^Azure\./". Please add "options.@azure-tools/typespec-csharp.package-dir" with expected value "/^Azure\./".
- Failed to find "options.@azure-tools/typespec-csharp.namespace" with expected value "{package-dir}". Please add "options.@azure-tools/typespec-csharp.namespace" with expected value "{package-dir}"..
- Failed to find "options.@azure-tools/typespec-csharp.clear-output-folder" with expected value "true". Please add "options.@azure-tools/typespec-csharp.clear-output-folder" with expected value "true".
- Failed to find "options.@azure-tools/typespec-csharp.package-dir" with expected value "/^Azure\.ResourceManager\./". Please add "options.@azure-tools/typespec-csharp.package-dir" with expected value "/^Azure\.ResourceManager\./".
```

hard to see from this which are are actually blocking, something like this could be nice

```
warning Validation on option "options.@azure-tools/typespec-csharp.package-dir" in "@azure-tools/typespec-csharp" are failed. However, per @azure-tools/typespec-csharp’s decision, we will treat it as passed.
warning Validation on option "options.@azure-tools/typespec-csharp.namespace" in "@azure-tools/typespec-csharp" are failed. However, per @azure-tools/typespec-csharp’s decision, we will treat it as passed.
warning Validation on option "options.@azure-tools/typespec-csharp.clear-output-folder" in "@azure-tools/typespec-csharp" are failed. However, per @azure-tools/typespec-csharp’s decision, we will treat it as passed.
warning Validation on option "options.@azure-tools/typespec-csharp.package-dir" in "@azure-tools/typespec-csharp" are failed. However, per @azure-tools/typespec-csharp’s decision, we will treat it as passed.

error Failed to find "options.@azure-tools/typespec-python.package-dir" with expected value "/^azure-mgmt(-[a-z]+){1,2}$/". Please add "options.@azure-tools/typespec-python.package-dir" with expected value "/^azure-mgmt(-[a-z]+){1,2}$/".
error Failed to find "options.@azure-tools/typespec-python.namespace" with expected value "/^azure\.mgmt(\.[a-z]+){1,2}$/". Please add "options.@azure-tools/typespec-python.namespace" with expected value "/^azure\.mgmt(\.[a-z]+){1,2}$/".
error Failed to find "options.@azure-tools/typespec-python.generate-sample" with expected value "true". Please add "options.@azure-tools/typespec-python.generate-sample" with expected value "true".
error Failed to find "options.@azure-tools/typespec-python.generate-test" with expected value "true". Please add "options.@azure-tools/typespec-python.generate-test" with expected value "true".
error Failed to find "options.@azure-tools/typespec-csharp.package-dir" with expected value "/^Azure\./". Please add "options.@azure-tools/typespec-csharp.package-dir" with expected value "/^Azure\./".
error Failed to find "options.@azure-tools/typespec-csharp.namespace" with expected value "{package-dir}". Please add "options.@azure-tools/typespec-csharp.namespace" with expected value "{package-dir}"..
error Failed to find "options.@azure-tools/typespec-csharp.clear-output-folder" with expected value "true". Please add "options.@azure-tools/typespec-csharp.clear-output-folder" with expected value "true".
error Failed to find "options.@azure-tools/typespec-csharp.package-dir" with expected value "/^Azure\.ResourceManager\./". Please add "options.@azure-tools/typespec-csharp.package-dir" with expected value "/^Azure\.ResourceManager\./".
```

combined with a custom problem matcher each error coudl show up in the right place directly in tspconfig.yaml in the pr review

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.