Investigate and polish perf in OpenAPI pipeline
- Dominant language
- C#
- Stars
- 38.4k
- Forks
- 10.9k
- Avg merge
- 2d 6h
- Merged PRs (30d)
- 290
Description
We introduced microbenchmarks to the new OpenAPI-generation implementation in preview4. Since then, we've introduced a few more features to the implementation that have perf impacts (more transformers, $ref support, recursive schema transformers). This issue tracks doing a perf analysis on the implementation after all these changes are in to assess impact and trade-offs of making improvements here.
Here's the baseline for numbers run against https://github.com/dotnet/aspnetcore/commit/28acf62c6f5d081fad40d7b0539bcf0b2cf81b17 for our current set of benchmarks.
| Method | EndpointCount | Mean | Error | StdDev | Op/s | Gen 0 | Gen 1 | Gen 2 | Allocated |
|----------------- |-------------- |------------:|----------:|----------:|---------:|---------:|--------:|------:|----------:|
| **GenerateDocument** | **10** | **366.4 μs** | **5.36 μs** | **5.74 μs** | **2,729.52** | **3.9063** | **-** | **-** | **694 KB** |
| **GenerateDocument** | **100** | **3,365.4 μs** | **29.89 μs** | **27.96 μs** | **297.14** | **46.8750** | **15.6250** | **-** | **6,295 KB** |
| **GenerateDocument** | **1000** | **37,951.0 μs** | **228.55 μs** | **202.61 μs** | **26.35** | **200.0000** | **-** | **-** | **62,314 KB** |
| Method | ElementCount | Mean | Error | StdDev | Op/s | Gen 0 | Gen 1 | Gen 2 | Allocated |
|-------------------------- |------------- |---------:|--------:|--------:|------------:|------:|------:|------:|----------:|
| OpenApiSchema_GetHashCode | 1 | 260.1 ns | 2.38 ns | 2.11 ns | 3,844,780.4 | - | - | - | - |
| OpenApiSchema_GetHashCode | 10 | 260.1 ns | 3.44 ns | 3.05 ns | 3,844,440.9 | - | - | - | - |
| OpenApiSchema_GetHashCode | 100 | 262.2 ns | 0.95 ns | 0.84 ns | 3,813,767.2 | - | - | - | - |
| Method | TransformerCount | Mean | Error | StdDev | Op/s | Gen 0 | Gen 1 | Gen 2 | Allocated |
|------------------------------- |----------------- |-----------:|----------:|----------:|----------:|-------:|-------:|------:|----------:|
| **ActivatedOperationTransformer** | **10** | **2.466 μs** | **0.0377 μs** | **0.0477 μs** | **405,514.3** | **0.0534** | **0.0114** | **-** | **7 KB** |
| OperationTransformerAsDelegate | 10 | 2.561 μs | 0.0195 μs | 0.0173 μs | 390,494.5 | 0.0496 | 0.0114 | - | 6 KB |
| ActivatedDocumentTransformer | 10 | 2.462 μs | 0.0369 μs | 0.0288 μs | 406,209.9 | 0.0534 | 0.0114 | - | 7 KB |
| DocumentTransformerAsDelegate | 10 | 2.399 μs | 0.0463 μs | 0.0455 μs | 416,841.0 | 0.0496 | 0.0114 | - | 6 KB |
| ActivatedSchemaTransformer | 10 | 31.617 μs | 0.4569 μs | 0.4274 μs | 31,628.4 | 0.3662 | - | - | 56 KB |
| SchemaTransformerAsDelegate | 10 | 30.972 μs | 0.6155 μs | 0.6320 μs | 32,287.4 | 0.3662 | - | - | 55 KB |
| **ActivatedOperationTransformer** | **100** | **4.082 μs** | **0.0557 μs** | **0.0494 μs** | **244,954.8** | **0.0687** | **0.0153** | **-** | **9 KB** |
| OperationTransformerAsDelegate | 100 | 3.250 μs | 0.0595 μs | 0.0497 μs | 307,655.7 | 0.0458 | 0.0076 | - | 6 KB |
| ActivatedDocumentTransformer | 100 | 4.149 μs | 0.0428 μs | 0.0358 μs | 241,048.5 | 0.0687 | 0.0153 | - | 9 KB |
| DocumentTransformerAsDelegate | 100 | 3.241 μs | 0.0258 μs | 0.0215 μs | 308,574.1 | 0.0496 | 0.0114 | - | 6 KB |
| ActivatedSchemaTransformer | 100 | 38.633 μs | 0.2470 μs | 0.2190 μs | 25,884.8 | 0.4883 | - | - | 66 KB |
| SchemaTransformerAsDelegate | 100 | 35.940 μs | 0.4064 μs | 0.3802 μs | 27,824.2 | 0.4883 | - | - | 62 KB |
| **ActivatedOperationTransformer** | **1000** | **34.187 μs** | **0.4856 μs** | **0.4543 μs** | **29,251.0** | **0.2441** | **-** | **-** | **30 KB** |
| OperationTransformerAsDelegate | 1000 | 11.128 μs | 0.0788 μs | 0.0658 μs | 89,863.8 | 0.0458 | 0.0153 | - | 6 KB |
| ActivatedDocumentTransformer | 1000 | 36.648 μs | 0.6589 μs | 0.8332 μs | 27,286.5 | 0.2441 | - | - | 30 KB |
| DocumentTransformerAsDelegate | 1000 | 12.282 μs | 0.0807 μs | 0.0674 μs | 81,422.7 | 0.0458 | 0.0153 | - | 6 KB |
| ActivatedSchemaTransformer | 1000 | 145.059 μs | 1.1951 μs | 0.9980 μs | 6,893.7 | 1.2207 | 0.2441 | - | 164 KB |
| SchemaTransformerAsDelegate | 1000 | 72.551 μs | 0.4466 μs | 0.3959 μs | 13,783.4 | 0.7324 | - | - | 117 KB |
Contributor guide
Assessment
This issue has not been assessed yet.