[CoreCLR/NativeAOT] Replace LLVM IR compilation with binary data packaging
- Dominant language
- C#
- Stars
- 2.1k
- Forks
- 579
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 257
Description
## Goal
Simplify the app build pipeline by eliminating LLVM IR code generation, `llc` compilation, and `ld` linking for CoreCLR and NativeAOT builds. This reduces build tool dependencies, build complexity, and long-term maintenance cost.
This must not come at the expense of a significant measurable startup performance regression. We need to measure the actual impact on real devices before and after.
## Summary
Every .NET for Android app build generates 5-7 LLVM IR (`.ll`) files per ABI, compiles them with `llc`, and links them with `ld` into `libxamarin-app.so`. This shared library is almost entirely **read-only data** — configuration structs, lookup tables, and pre-allocated buffers. The LLVM IR pipeline is a heavyweight code generation + compilation step for what is fundamentally a data packaging problem.
We already have a simpler mechanism for packaging data into the APK: `DSOWrapperGenerator` wraps arbitrary binary files in a minimal ELF `.so` using `llvm-objcopy`, places them in `lib/{abi}/`, and the runtime mmaps them directly from the APK. This is how assembly stores (`assemblies.blob`) work today.
**Proposal:** Replace the LLVM IR → `llc` → `ld` pipeline with direct binary serialization → `llvm-objcopy` for all configuration data. The existing `LlvmIrComposer` subclasses already compute all the values in C# — we just change the output stage from "emit LLVM IR text" to "write raw bytes matching the C struct layout."
**Scope:** CoreCLR and NativeAOT only. MonoVM continues using LLVM IR until deprecated.
**Dependency:** Trimmable TypeMap (in progress) eliminates `typemaps` and `marshal_methods`. This proposal handles the remaining `.ll` files.
## What's in `libxamarin-app.so` today
| File | Contents | Replacement |
|---|---|---|
| `typemaps.*.ll` | Java↔.NET type mapping tables | Trimmable TypeMap (in progress) |
| `marshal_methods.*.ll` | Marshal method init stub | Trimmable TypeMap (in progress) |
| `environment.*.ll` | `ApplicationConfig` struct, runtime properties, DSO cache, env vars | **Binary blob** (this proposal) |
| `compressed_assemblies.*.ll` | Decompression descriptors + zero-init buffer | **Binary blob** + dynamic alloc |
| `jni_remap.*.ll` | JNI type/method remapping (Intune MAM) | **Binary blob** or managed Dictionary |
| `pinvoke_preserve.*.ll` | P/Invoke symbol preservation + `find_pinvoke()` (CoreCLR unified linking only) | **Linker flags** (this proposal) |
| `jni_init_funcs.*.ll` | NativeAOT: JNI_OnLoad dispatch | **Generated C#** (this proposal) |
## Proposed approach
### 1. Binary config blob in ELF wrapper (replaces `environment`, `compressed_assemblies`, `jni_remap`)
**Build time:**
The existing `LlvmIrComposer` subclasses (e.g., `ApplicationConfigNativeAssemblyGeneratorCLR`, `CompressedAssembliesNativeAssemblyGenerator`) already have a two-stage pipeline:
1. **Compose** — compute all values, populate `StructureInstance`, `List>`, etc.
2. **Generate** — serialize to LLVM IR text via `LlvmIrGenerator`
We replace stage 2: instead of `LlvmIrGenerator` emitting text, a new `BinaryBlobWriter` serializes `StructureInstance` objects directly to bytes using the existing `StructureInfo` metadata (field offsets, sizes, alignment, padding). Same data, same layout, no compilation step.
The output `config.bin` has a simple section-based format:
```
[Header: magic, version, section_count, section_offsets[], section_sizes[]]
[Section 0: ApplicationConfig] // binary-compatible with C struct
[Section 1: runtime property names] // null-terminated string table
[Section 2: runtime property values] // null-terminated string table
[Section 3: DSOCacheEntry[]] // array of C structs
[Section 4: DSO name string data]
[Section 5: DSOApkEntry[]] // template entries (fd filled at runtime)
[Section 6: compressed assembly descriptors]
```
Then wrap and package:
```
DSOWrapperGenerator.WrapIt(config.bin) → lib/{abi}/libruntime-config.so
```
This uses `llvm-objcopy --add-section payload=config.bin` — the same tool already used for assembly stores. No `llc`, no `ld`.
**Runtime:**
The zip scan (which already runs to find assembly stores) discovers `libruntime-config.so`, mmaps it from the APK, and `get_wrapper_dso_payload_pointer_and_size()` returns a direct pointer to the payload.
```cpp
// Same infrastructure as assembly store loading
auto [data, size] = get_wrapper_dso_payload_pointer_and_size(mmap_info, "libruntime-config.so");
// Parse header, cast section pointers directly to C structs
auto header = static_cast(data);
auto base = static_cast(data);
application_config = reinterpret_cast(base + header->sections[0].offset);
dso_cache = reinterpret_cast(base + header->sections[3].offset);
```
No parsing, no copying, no deserialization. The MSBuild task writes bytes matching the C struct memory layout. The C++ code casts pointers into the mmap'd region. Data is demand-paged from the APK by the kernel — same mechanism as today.
**Changes to `libmonodroid.so`:** Replace `extern` declarations (currently resolved by `libxamarin-app.so` at load time) with static pointer globals initialized from the mmap'd blob. This follows the same pattern already used for `assembly_store` data.
### 2. Dynamic allocation (replaces BSS pre-allocated buffers)
The zero-initialized buffers in `libxamarin-app.so` (assembly store slots, decompression buffer) use BSS sections, which the kernel backs with `mmap(MAP_ANONYMOUS)` + demand paging. Allocating with `new[]` uses the same kernel mechanism for large allocations. Replace:
```cpp
// Before: LLVM IR pre-allocates in BSS
extern uint8_t uncompressed_assemblies_data_buffer[];
extern AssemblyStoreSingleAssemblyRuntimeData assembly_store_bundled_assemblies[];
// After: allocate at startup (size from config blob)
auto buffer = new uint8_t[config->total_uncompressed_size]();
auto assemblies = new AssemblyStoreSingleAssemblyRuntimeData[config->assembly_count]();
```
### 3. Environment variables → Java `Os.setenv()`
Generate Java code calling `Os.setenv()` before `initInternal()`, following the pattern NativeAOT already uses (`NativeAotEnvironmentVars.java`).
### 4. `pinvoke_preserve.*.ll` → Linker flags + `dlsym` (CoreCLR unified linking only)
This is the one file with actual executable code: `find_pinvoke()` maps `(library_hash, entrypoint_hash)` → function pointer via nested switch statements. It serves two purposes:
**Linker symbol preservation** — references to symbols like `@SystemNative_Bind` prevent `--gc-sections` from stripping them. Replace with `--undefined=` linker flags. `PinvokeScanner` already produces the symbol list, and `NativeLinker.cs` already supports `--export-dynamic-symbol` — the infrastructure is in place. (There's even a TODO in `dynamic.cc:88` where the team considered this approach.)
**Runtime P/Invoke resolution** — replace with `dlsym(RTLD_DEFAULT, entrypoint_name)`, which already exists as a fallback in `dynamic.cc`. P/Invoke results are cached by CoreCLR — each entrypoint is resolved once. Performance impact to be measured.
### 5. `jni_init_funcs.*.ll` → Generated C# (NativeAOT only)
Replace the LLVM IR function pointer array with generated C# using `[DllImport("__Internal")]`:
```csharp
static class JniInitFunctions
{
[DllImport("__Internal")]
static extern int JNI_OnLoad_SystemNative (IntPtr vm, IntPtr reserved);
[DllImport("__Internal")]
static extern int JNI_OnLoad_CryptoNative (IntPtr vm, IntPtr reserved);
public static void CallAll (IntPtr vm)
{
JNI_OnLoad_SystemNative (vm, IntPtr.Zero);
JNI_OnLoad_CryptoNative (vm, IntPtr.Zero);
}
}
```
NativeAOT compiles `[DllImport("__Internal")]` to direct native call instructions — zero overhead, compile-time symbol resolution, missing symbol = link error (not runtime crash). The `DirectPInvoke` infrastructure already exists in `Microsoft.Android.Sdk.NativeAOT.targets`.
## Performance
The primary goal is long-term maintainability and build simplification. However, this must not come at the expense of a significant measurable startup regression.
The proposed approach uses the same mmap-from-APK mechanism as today — config data is still accessed via direct pointer dereferences into memory-mapped regions. The main differences are: (1) eliminating `dlopen("libxamarin-app.so")` and its symbol resolution overhead, (2) replacing BSS pre-allocated buffers with dynamic `new[]`, and (3) replacing `find_pinvoke()` with `dlsym` for unified linking.
**We need to measure the actual performance impact** on real devices (high-end and low-end) with representative apps before and after. A feature flag should allow A/B comparison.
### Build time
| Current | Proposed |
|---|---|
| 5-7 `llc` invocations per ABI (LLVM IR compilation) | Eliminated |
| 1 `ld` invocation per ABI (native linking) | Eliminated |
| — | 1 `llvm-objcopy` per ABI (already used for assembly stores) |
## Work items
### Phase 1: Binary blob infrastructure
- [ ] `BinaryBlobWriter`: serialize `StructureInstance` to raw bytes using `StructureInfo` metadata
- [ ] Define blob header format (magic, version, section table)
- [ ] MSBuild task: reuse existing `LlvmIrComposer.Compose()` → `BinaryBlobWriter` → `DSOWrapperGenerator.WrapIt()`
- [ ] C++ `init_runtime_config()`: mmap blob from APK, parse header, set global pointers
- [ ] Convert `extern` declarations in `xamarin-app.hh` to static pointer globals (gated on MonoVM compat)
### Phase 2: Migrate data (incremental, per section)
- [ ] `ApplicationConfig` struct
- [ ] Runtime properties (name/value string tables for `coreclr_initialize()`)
- [ ] DSO cache + APK entries + name data
- [ ] Compressed assembly descriptors
- [ ] Pre-allocated buffers → dynamic `new[]`
- [ ] Environment variables → Java `Os.setenv()`
- [ ] JNI remapping tables
### Phase 3: Executable code replacements
- [ ] `pinvoke_preserve.ll` → `--undefined` linker flags + `dlsym(RTLD_DEFAULT)`
- [ ] `jni_init_funcs.ll` (NativeAOT) → generated C# with `[DllImport("__Internal")]`
- [ ] NativeAOT `environment.ll` → generated Java or C#
### Phase 4: Cleanup
- [ ] Remove `System.loadLibrary("xamarin-app")` for CoreCLR/NativeAOT
- [ ] Gate LLVM IR generators to MonoVM-only
- [ ] Remove `libxamarin-app.so` from CoreCLR/NativeAOT APK
- [ ] Gate `llc`/`ld` to MonoVM builds only
## Risks and mitigations
| Risk | Mitigation |
|---|---|
| Startup regression | Benchmark on real devices before/after. Feature flag for A/B. Old path remains until validated. |
| Struct layout drift (MSBuild writer vs C++ reader) | Reuse existing `StructureInfo` metadata for binary layout. Version header enables forward compat. |
| MonoVM compatibility | All changes gated behind runtime check. MonoVM path unchanged. |
| Desktop designer | `application_dso_stub.cc` remains. |
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.