apache / apache/pulsar-dotpulsar
AOT (Ahead-of-Time) Compilation Compatibility Gaps
- Dominant language
- C#
- Stars
- 271
- Forks
- 87
- PR merge metrics
- No merged PRs in 30d
Description
### Description
DotPulsar is not compatible with .NET Native AOT publishing. Attempting to use it in a project built with `true` produces multiple trim and dynamic-code warnings at build time, and the produced binary may crash at runtime when compression or schema features are exercised.
The root causes fall into three areas:
1. **Compression plugins** — all five compression backends (`BuiltinZlibCompression`, `Lz4Compression`, `SnappyCompression`, `ZstdCompression`, `ZstdSharpCompression`) discover and invoke the compression libraries entirely through `Assembly.Load()` + runtime reflection. These patterns are fundamentally incompatible with AOT and trimming.
2. **Avro schemas** — `AvroISpecificRecordSchema` and `AvroGenericRecordSchema` also use `Assembly.Load("Avro")`, `DefinedTypes`, `MakeGenericType()`, and `Activator.CreateInstance()` to bind to the Apache Avro library at runtime.
3. **JSON schema** — `JsonSchemaDefinitionGenerator` inspects types via `GetProperties()` / `GetCustomAttribute()` / `Enum.GetNames()` at runtime, and `JsonSchema` calls `JsonSerializer.Deserialize` / `SerializeToUtf8Bytes` without a source-generated `JsonSerializerContext`, which is trim-unsafe.
The library also declares no AOT-compatibility stance (`` is absent from the `.csproj`) and carries no `[RequiresDynamicCode]` / `[RequiresUnreferencedCode]` annotations to alert consumers.
### Reproduction Steps
1. Create a new .NET 8/9/10 console application.
2. Reference DotPulsar (latest, 5.3.1 at time of writing).
3. Add `true` to the project file.
4. Call any API that touches compression or a JSON/Avro schema (or simply add `using DotPulsar;`).
5. Publish: `dotnet publish -r linux-x64 -c Release`
Build output (verified with `dotnet publish -r win-x64 -c Release` against DotPulsar 5.3.1 / .NET 9 SDK) includes **39 warnings** — a representative sample:
```
AvroISpecificRecordSchema.cs(114): Trim analysis warning IL2026: ...Assembly.DefinedTypes.get which has 'RequiresUnreferencedCodeAttribute'...
AvroISpecificRecordSchema.cs(136): AOT analysis warning IL3050: ...System.Type.MakeGenericType(Type[]) which has 'RequiresDynamicCodeAttribute'...
AvroISpecificRecordSchema.cs(136): Trim analysis warning IL2055: Call to 'System.Type.MakeGenericType(Type[])' can not be statically analyzed...
BuiltinZlibCompression.cs(28): Trim analysis warning IL2026: ...Assembly.GetTypes() which has 'RequiresUnreferencedCodeAttribute'...
BuiltinZlibCompression.cs(35): AOT analysis warning IL3050: ...System.Type.GetEnumValues() which has 'RequiresDynamicCodeAttribute'...
Lz4Compression.cs(34): Trim analysis warning IL2026: ...Assembly.DefinedTypes.get which has 'RequiresUnreferencedCodeAttribute'...
SnappyCompression.cs(32): Trim analysis warning IL2026: ...Assembly.DefinedTypes.get which has 'RequiresUnreferencedCodeAttribute'...
ZstdCompression.cs(33): Trim analysis warning IL2026: ...Assembly.DefinedTypes.get which has 'RequiresUnreferencedCodeAttribute'...
ZstdSharpCompression.cs(33): Trim analysis warning IL2026: ...Assembly.DefinedTypes.get which has 'RequiresUnreferencedCodeAttribute'...
JsonSchemaDefinitionGenerator.cs(53): Trim analysis warning IL2070: ...'this' argument does not satisfy 'PublicProperties' in call to 'System.Type.GetProperties(BindingFlags)'...
```
The full list is in the **Other information** section below.
At runtime (depending on trimming aggressiveness), the compressor factory initialization will throw `FileNotFoundException` (trimmed assembly) or `MissingMethodException`.
### Expected behavior
The library should either:
- Be fully AOT-compatible for the core messaging path (at minimum) and annotate any optional features (Avro, dynamic compression loading) with `[RequiresDynamicCode]` / `[RequiresUnreferencedCode]` so the compiler can warn consumers; **or**
- Declare `false` in the `.csproj` and document the limitation.
Ideally the core path (connect, produce, consume with `byte[]` / primitive schemas) would be AOT-safe, with opt-in AOT-unsafe features clearly annotated.
### Actual behavior
All five compression loaders use `Assembly.Load()` and extensive reflection with no annotations. The Avro schema implementations do the same. The JSON schema implementation uses un-annotated `System.Text.Json` reflection APIs. The project carries zero `[RequiresDynamicCode]` / `[RequiresUnreferencedCode]` attributes and no `` marker.
### Regression?
Not a regression — AOT compatibility has not been declared or targeted in any prior release.
### Known Workarounds
- Build without `` and instead use `` with `true` (dynamic loading still works).
- Suppress individual warnings with `` / `` in the consuming project (fragile, requires intimate knowledge of the library internals).
- For the JSON schema specifically, pass a pre-built `jsonSchemaDefinition` string to `JsonSchema(options, jsonSchemaDefinition)` and supply a source-generated `JsonSerializerContext` via `JsonSerializerOptions`.
### Configuration
- DotPulsar version: 5.3.1
- .NET SDK: 9.0 / 10.0 (both affected)
- OS: Windows 11, Linux x64 (both affected)
- Architecture: x64, ARM64
### Other information
### Detailed breakdown of AOT-incompatible code
#### 1. Compression — `Assembly.Load()` + runtime reflection
All five files under `src/DotPulsar/Internal/Compression/` follow the same pattern: load an optional assembly at runtime, enumerate its types by full name, and create delegates or instances via `Activator.CreateInstance` / `MethodInfo.CreateDelegate`. None of these patterns survive trimming.
| File | Key problematic APIs |
|------|----------------------|
| `BuiltinZlibCompression.cs:27` | `Assembly.Load("System.IO.Compression")`, `assembly.GetTypes()`, `Activator.CreateInstance(zlibStreamType!, ...)` (lines 60, 71) |
| `Lz4Compression.cs:32` | `Assembly.Load("K4os.Compression.LZ4")`, `assembly.DefinedTypes.ToArray()`, `lz4Codec.GetMethods(BindingFlags.Public \| BindingFlags.Static)`, `method.CreateDelegate(...)` |
| `SnappyCompression.cs:30` | `Assembly.Load("IronSnappy")`, `snappy.GetMethods(BindingFlags.Public \| BindingFlags.Static)`, `method.CreateDelegate(...)` |
| `ZstdCompression.cs:31` | `Assembly.Load("ZstdNet")`, `GetMethods(BindingFlags.Public \| BindingFlags.Instance)`, `Activator.CreateInstance(...)`, `wrapMethod.CreateDelegate(...)` |
| `ZstdSharpCompression.cs:31` | `Assembly.Load("ZstdSharp")`, same pattern as above |
**Suggested fix:** Take compile-time dependencies on the optional compression packages instead of loading them dynamically. Gate each implementation behind a `#if` / multi-targeting or expose a registration API (`builder.UseSnappy()`) that accepts a concrete `ICompressorFactory` / `IDecompressorFactory` supplied by the consumer. This is the pattern used by libraries like `Microsoft.Extensions.Caching.StackExchangeRedis`.
#### 2. Avro schemas — `Assembly.Load()` + `MakeGenericType` + `Activator.CreateInstance`
`AvroISpecificRecordSchema` (lines 54, 57, 59, 71–72, 113–114, 119–120, 136, 149, 156, 159) and `AvroGenericRecordSchema` use an identical late-binding strategy against the `Apache.Avro` NuGet package.
Notable calls:
- `Assembly.Load("Avro")` — the assembly is trimmed away unless explicitly preserved
- `assembly.DefinedTypes.ToArray()` — all types must survive trimming for this to work
- `type.MakeGenericType(typeof(T))` — `[RequiresDynamicCode]`
- `Activator.CreateInstance(_avroWriterTypeInfo, _avroSchema)` — `[RequiresUnreferencedCode]`
- `MethodInfo.Invoke(...)` in `Decode` / `Encode` hot paths — defeats inlining and is trim-unsafe
**Suggested fix:** Take a direct compile-time reference to `Apache.Avro` and use the concrete `SpecificDatumWriter` / `SpecificDatumReader` types. If the dependency must remain optional, annotate both schema classes with `[RequiresDynamicCode("Avro schema binding uses runtime reflection.")]` and `[RequiresUnreferencedCode(...)]`.
#### 3. `JsonSchemaDefinitionGenerator` — runtime type inspection
`src/DotPulsar/Schemas/JsonSchemaDefinitionGenerator.cs` calls `GetProperties(BindingFlags.Public | BindingFlags.Instance)` (line 53), `GetCustomAttribute()` (line 58), `GetCustomAttribute()` (line 81), `Enum.GetNames(type)` (line 135), `type.GetElementType()` (line 154), `type.GetGenericTypeDefinition()` (line 161), and `type.GetGenericArguments()` (lines 170, 178) — all trim-unsafe when the concrete `T` is not statically known.
**Suggested fix:** Replace with a `System.Text.Json` source-generated schema approach (available in .NET 9 via `JsonSchemaExporter` in `System.Text.Json`) or accept an explicit schema string and deprecate the reflection-based generator. Short-term: annotate `Generate(Type, ...)` with `[RequiresUnreferencedCode]`.
#### 4. `JsonSchema` — trim-unsafe `JsonSerializer` calls
`src/DotPulsar/Schemas/JsonSchema.cs` calls `JsonSerializer.Deserialize(array, _options)` (line 59) and `JsonSerializer.SerializeToUtf8Bytes(message, _options)` (line 70). Both overloads without a `JsonTypeInfo` parameter are decorated with `[RequiresUnreferencedCode]` in the BCL.
**Suggested fix:** Add an overload that accepts `JsonTypeInfo` (or a `JsonSerializerContext`) and route serialization through it. Mark the existing reflection-based constructors with `[RequiresUnreferencedCode]`.
#### 5. No `` declaration
The absence of `true` in `DotPulsar.csproj` means the .NET SDK trim analyzer is **not** run during normal builds of the library itself. Adding `true` (even as a goal/aspiration) would surface all of the above as build-time errors in the library's own CI, making it impossible to accidentally re-introduce these patterns.
---
### Full verified warning list
Obtained by running `dotnet publish -r win-x64 -c Release` against a minimal consumer project referencing DotPulsar 5.3.1 source (the native link step fails because MSVC is not on `PATH`, but all ILLink/AOT analyzer warnings are emitted before that step).
**39 warnings across 8 source files:**
| # | File | Line | Warning | Description |
|---|------|------|---------|-------------|
| 1 | `Schemas/AvroISpecificRecordSchema.cs` | 54 | IL2080 | `GetInterfaces()` — field `_typeT` not annotated with `DynamicallyAccessedMemberTypes.Interfaces` |
| 2 | `Schemas/AvroISpecificRecordSchema.cs` | 57 | IL2080 | `GetField()` — field `_typeT` not annotated with `PublicFields` |
| 3 | `Schemas/AvroISpecificRecordSchema.cs` | 71 | IL2075 | `GetProperty()` — return of `GetType()` not annotated with `PublicProperties` |
| 4 | `Schemas/AvroISpecificRecordSchema.cs` | 72 | IL2075 | `GetMethod()` — return of `GetType()` not annotated with `PublicMethods` |
| 5 | `Schemas/AvroISpecificRecordSchema.cs` | 114 | IL2026 | `Assembly.DefinedTypes` has `[RequiresUnreferencedCode]` — types may be removed |
| 6 | `Schemas/AvroISpecificRecordSchema.cs` | 119 | IL2065 | `GetMethods()` — `this` value not statically determinable |
| 7 | `Schemas/AvroISpecificRecordSchema.cs` | 120 | IL2065 | `GetMethods()` — same as above (second call) |
| 8 | `Schemas/AvroISpecificRecordSchema.cs` | 136 | IL3050 | `MakeGenericType()` has `[RequiresDynamicCode]` — native code for instantiation may be absent |
| 9 | `Schemas/AvroISpecificRecordSchema.cs` | 136 | IL2055 | `MakeGenericType()` — call cannot be statically analyzed |
| 10 | `Schemas/AvroISpecificRecordSchema.cs` | 149 | IL3050 | `MakeGenericType()` — same (reader type) |
| 11 | `Schemas/AvroISpecificRecordSchema.cs` | 149 | IL2055 | `MakeGenericType()` — same (reader type) |
| 12 | `Schemas/AvroISpecificRecordSchema.cs` | 156 | IL2077 | `Activator.CreateInstance()` — field `_avroWriterTypeInfo` not annotated with `PublicConstructors` |
| 13 | `Schemas/AvroISpecificRecordSchema.cs` | 159 | IL2077 | `Activator.CreateInstance()` — field `_avroReaderTypeInfo` not annotated with `PublicConstructors` |
| 14 | `Internal/Compression/BuiltinZlibCompression.cs` | 28 | IL2026 | `Assembly.GetTypes()` has `[RequiresUnreferencedCode]` |
| 15 | `Internal/Compression/BuiltinZlibCompression.cs` | 35 | IL3050 | `Type.GetEnumValues()` has `[RequiresDynamicCode]` |
| 16 | `Internal/Compression/BuiltinZlibCompression.cs` | 37 | IL3050 | `Type.GetEnumValues()` — same (second enum type) |
| 17 | `Internal/Compression/BuiltinZlibCompression.cs` | 60 | IL2067 | `Activator.CreateInstance()` — parameter `zlibStreamType` not annotated with `PublicConstructors` |
| 18 | `Internal/Compression/BuiltinZlibCompression.cs` | 71 | IL2067 | `Activator.CreateInstance()` — same (decompressor lambda) |
| 19 | `Internal/Compression/Lz4Compression.cs` | 34 | IL2026 | `Assembly.DefinedTypes` has `[RequiresUnreferencedCode]` |
| 20 | `Internal/Compression/Lz4Compression.cs` | 39 | IL2075 | `GetMethods()` — return of `FindLZ4Codec()` not annotated with `PublicMethods` |
| 21 | `Internal/Compression/SnappyCompression.cs` | 32 | IL2026 | `Assembly.DefinedTypes` has `[RequiresUnreferencedCode]` |
| 22 | `Internal/Compression/SnappyCompression.cs` | 36 | IL2075 | `GetMethods()` — return of `FindSnappy()` not annotated with `PublicMethods` |
| 23 | `Internal/Compression/ZstdCompression.cs` | 33 | IL2026 | `Assembly.DefinedTypes` has `[RequiresUnreferencedCode]` |
| 24 | `Internal/Compression/ZstdCompression.cs` | 36 | IL2075 | `GetMethods()` — return of `Find()` not annotated with `PublicMethods` |
| 25 | `Internal/Compression/ZstdCompression.cs` | 40 | IL2075 | `GetMethods()` — return of `Find()` not annotated with `PublicMethods` (second call) |
| 26 | `Internal/Compression/ZstdCompression.cs` | 45 | IL2072 | `Activator.CreateInstance()` — return of `Find()` not annotated with `PublicParameterlessConstructor` |
| 27 | `Internal/Compression/ZstdCompression.cs` | 55 | IL2072 | `Activator.CreateInstance()` — same (decompressor lambda) |
| 28 | `Internal/Compression/ZstdCompression.cs` | 83 | IL2075 | `TypeInfo.ImplementedInterfaces` — enumerator current not annotated with `Interfaces` |
| 29 | `Internal/Compression/ZstdCompression.cs` | 83 | IL2075 | `Type.GetConstructor()` — enumerator current not annotated with `PublicParameterlessConstructor` |
| 30 | `Internal/Compression/ZstdSharpCompression.cs` | 33 | IL2026 | `Assembly.DefinedTypes` has `[RequiresUnreferencedCode]` |
| 31 | `Internal/Compression/ZstdSharpCompression.cs` | 36 | IL2075 | `GetMethods()` — return of `FindDecompressor()` not annotated with `PublicMethods` |
| 32 | `Internal/Compression/ZstdSharpCompression.cs` | 40 | IL2075 | `GetMethods()` — return of `FindCompressor()` not annotated with `PublicMethods` |
| 33 | `Internal/Compression/ZstdSharpCompression.cs` | 45 | IL2072 | `Activator.CreateInstance()` — return of `FindCompressor()` not annotated with `PublicConstructors` |
| 34 | `Internal/Compression/ZstdSharpCompression.cs` | 55 | IL2072 | `Activator.CreateInstance()` — return of `FindDecompressor()` not annotated with `PublicParameterlessConstructor` |
| 35 | `Internal/Compression/ZstdSharpCompression.cs` | 83 | IL2075 | `TypeInfo.ImplementedInterfaces` — enumerator current not annotated with `Interfaces` (FindDecompressor) |
| 36 | `Internal/Compression/ZstdSharpCompression.cs` | 83 | IL2075 | `Type.GetConstructor()` — enumerator current not annotated with `PublicParameterlessConstructor` (FindDecompressor) |
| 37 | `Internal/Compression/ZstdSharpCompression.cs` | 103 | IL2075 | `TypeInfo.ImplementedInterfaces` — enumerator current not annotated with `Interfaces` (FindCompressor) |
| 38 | `Internal/Compression/ZstdSharpCompression.cs` | 103 | IL2075 | `Type.GetConstructor()` — enumerator current not annotated with `PublicConstructors` (FindCompressor) |
| 39 | `Schemas/JsonSchemaDefinitionGenerator.cs` | 53 | IL2070 | `GetProperties(BindingFlags)` — parameter `type` not annotated with `PublicProperties` |
> **Note:** `AvroGenericRecordSchema` and the `JsonSchema.Decode` / `Encode` paths were not reachable from the probe's entry point, so their warnings do not appear in this run. The actual warning count in a real application touching those code paths will be higher.
Contributor guide
Research direction
Start by reproducing the warnings with `dotnet publish -r win-x64 -c Release`, then inspect the five files under `src/DotPulsar/Internal/Compression/`, the Avro schema files, `JsonSchemaDefinitionGenerator.cs`, `JsonSchema.cs`, and `DotPulsar.csproj`. Compare the possible AOT compatibility approaches in the issue and define done as a documented compatibility stance, addressed or explicitly annotated warnings, and a working core messaging path under Native AOT.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp
- Domain
- backend, build-system
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100