Improve the UX of api-diff suppression baseline and test resiliency
- Dominant language
- No language data
- Stars
- 287
- Forks
- 145
- Avg merge
- 1d 22h
- Merged PRs (30d)
- 10
Description
With https://github.com/dotnet/dotnet/pull/2542 we have enabled api-diffs between Microsoft and source-build SDKs. Resulting diffs are captured in https://github.com/dotnet/source-build/issues/5344
The implementation uses a built-in feature of ApiCompat task to generate a new suppression baseline, which is further compared with the old baseline. The issue exists with UX of the baseline file. It is not clear which assembly is missing the API, so in the following baseline entry example either `left` or `right` assembly are missing the API:
```
CP0001
T:System.Diagnostics.CodeAnalysis.ExperimentalAttribute
/tmp/ak2gfs0d.ycb/msft/sdk/10.0.100-rc.2.25465.105/DotnetTools/dotnet-watch/10.0.100-rc.2.25465.105/tools/net10.0/any/BuildHost-netcore/Microsoft.CodeAnalysis.Workspaces.MSBuild.BuildHost.dll
/tmp/ak2gfs0d.ycb/sb/sdk/10.0.100-rc.2.25465.105/DotnetTools/dotnet-watch/10.0.100-rc.2.25465.105/tools/net10.0/any/BuildHost-netcore/Microsoft.CodeAnalysis.Workspaces.MSBuild.BuildHost.dll
```
Unlike suppression file, the task's output is specific in calling out the actual assembly missing the type:
```
error CP0001: Type 'System.Diagnostics.CodeAnalysis.ExperimentalAttribute' exists on /tmp/zezs3i1y.dqa/sb/sdk/10.0.100-rc.2.25465.105/DotnetTools/dotnet-watch/10.0.100-rc.2.25465.105/tools/net10.0/any/BuildHost-netcore/Microsoft.CodeAnalysis.Workspaces.MSBuild.BuildHost.dll but not on /tmp/zezs3i1y.dqa/msft/sdk/10.0.100-rc.2.25465.105/DotnetTools/dotnet-watch/10.0.100-rc.2.25465.105/tools/net10.0/any/BuildHost-netcore/Microsoft.CodeAnalysis.Workspaces.MSBuild.BuildHost.dll
```
Additionally, the current model would not capture potential bugs, where a specific API, in a single change, disappears on one binary and appears on the other. This can happen if some APIs are conditioned-out on source-build and someone accidentally alters this condition by simply reversing it. The API would disappear from MSFT build, appear in source-build, but the suppression file would not have any differences, thus the test would not fail.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.