dotnet / dotnet/android

[CoreCLR/NativeAOT] Replace LLVM IR compilation with binary data packaging

Open
#10,784 4 comments 0 reactions 0 assignees View on GitHub
Area: CoreCLR Area: NativeAOT
Dominant language
C#
Stars
2.1k
Forks
579
Avg merge
1d 21h
Merged PRs (30d)
257

Description

## Goal

Simplify the app build pipeline by eliminating LLVM IR code generation, `llc` compilation, and `ld` linking for CoreCLR and NativeAOT builds. This reduces build tool dependencies, build complexity, and long-term maintenance cost.

This must not come at the expense of a significant measurable startup performance regression. We need to measure the actual impact on real devices before and after.

## Summary

Every .NET for Android app build generates 5-7 LLVM IR (`.ll`) files per ABI, compiles them with `llc`, and links them with `ld` into `libxamarin-app.so`. This shared library is almost entirely **read-only data** — configuration structs, lookup tables, and pre-allocated buffers. The LLVM IR pipeline is a heavyweight code generation + compilation step for what is fundamentally a data packaging problem.

We already have a simpler mechanism for packaging data into the APK: `DSOWrapperGenerator` wraps arbitrary binary files in a minimal ELF `.so` using `llvm-objcopy`, places them in `lib/{abi}/`, and the runtime mmaps them directly from the APK. This is how assembly stores (`assemblies.blob`) work today.

**Proposal:** Replace the LLVM IR → `llc` → `ld` pipeline with direct binary serialization → `llvm-objcopy` for all configuration data. The existing `LlvmIrComposer` subclasses already compute all the values in C# — we just change the output stage from "emit LLVM IR text" to "write raw bytes matching the C struct layout."

**Scope:** CoreCLR and NativeAOT only. MonoVM continues using LLVM IR until deprecated.

**Dependency:** Trimmable TypeMap (in progress) eliminates `typemaps` and `marshal_methods`. This proposal handles the remaining `.ll` files.

## What's in `libxamarin-app.so` today

| File | Contents | Replacement |
|---|---|---|
| `typemaps.*.ll` | Java↔.NET type mapping tables | Trimmable TypeMap (in progress) |
| `marshal_methods.*.ll` | Marshal method init stub | Trimmable TypeMap (in progress) |
| `environment.*.ll` | `ApplicationConfig` struct, runtime properties, DSO cache, env vars | **Binary blob** (this proposal) |
| `compressed_assemblies.*.ll` | Decompression descriptors + zero-init buffer | **Binary blob** + dynamic alloc |
| `jni_remap.*.ll` | JNI type/method remapping (Intune MAM) | **Binary blob** or managed Dictionary |
| `pinvoke_preserve.*.ll` | P/Invoke symbol preservation + `find_pinvoke()` (CoreCLR unified linking only) | **Linker flags** (this proposal) |
| `jni_init_funcs.*.ll` | NativeAOT: JNI_OnLoad dispatch | **Generated C#** (this proposal) |

## Proposed approach

### 1. Binary config blob in ELF wrapper (replaces `environment`, `compressed_assemblies`, `jni_remap`)

**Build time:**

The existing `LlvmIrComposer` subclasses (e.g., `ApplicationConfigNativeAssemblyGeneratorCLR`, `CompressedAssembliesNativeAssemblyGenerator`) already have a two-stage pipeline:

1. **Compose** — compute all values, populate `StructureInstance`, `List>`, etc.
2. **Generate** — serialize to LLVM IR text via `LlvmIrGenerator`

We replace stage 2: instead of `LlvmIrGenerator` emitting text, a new `BinaryBlobWriter` serializes `StructureInstance` objects directly to bytes using the existing `StructureInfo` metadata (field offsets, sizes, alignment, padding). Same data, same layout, no compilation step.

The output `config.bin` has a simple section-based format:

```
[Header: magic, version, section_count, section_offsets[], section_sizes[]]
[Section 0: ApplicationConfig] // binary-compatible with C struct
[Section 1: runtime property names] // null-terminated string table
[Section 2: runtime property values] // null-terminated string table
[Section 3: DSOCacheEntry[]] // array of C structs
[Section 4: DSO name string data]
[Section 5: DSOApkEntry[]] // template entries (fd filled at runtime)
[Section 6: compressed assembly descriptors]
```

Then wrap and package:

```
DSOWrapperGenerator.WrapIt(config.bin) → lib/{abi}/libruntime-config.so
```

This uses `llvm-objcopy --add-section payload=config.bin` — the same tool already used for assembly stores. No `llc`, no `ld`.

**Runtime:**

The zip scan (which already runs to find assembly stores) discovers `libruntime-config.so`, mmaps it from the APK, and `get_wrapper_dso_payload_pointer_and_size()` returns a direct pointer to the payload.

```cpp
// Same infrastructure as assembly store loading
auto [data, size] = get_wrapper_dso_payload_pointer_and_size(mmap_info, "libruntime-config.so");

// Parse header, cast section pointers directly to C structs
auto header = static_cast(data);
auto base = static_cast(data);
application_config = reinterpret_cast(base + header->sections[0].offset);
dso_cache = reinterpret_cast(base + header->sections[3].offset);
```

No parsing, no copying, no deserialization. The MSBuild task writes bytes matching the C struct memory layout. The C++ code casts pointers into the mmap'd region. Data is demand-paged from the APK by the kernel — same mechanism as today.

**Changes to `libmonodroid.so`:** Replace `extern` declarations (currently resolved by `libxamarin-app.so` at load time) with static pointer globals initialized from the mmap'd blob. This follows the same pattern already used for `assembly_store` data.

### 2. Dynamic allocation (replaces BSS pre-allocated buffers)

The zero-initialized buffers in `libxamarin-app.so` (assembly store slots, decompression buffer) use BSS sections, which the kernel backs with `mmap(MAP_ANONYMOUS)` + demand paging. Allocating with `new[]` uses the same kernel mechanism for large allocations. Replace:

```cpp
// Before: LLVM IR pre-allocates in BSS
extern uint8_t uncompressed_assemblies_data_buffer[];
extern AssemblyStoreSingleAssemblyRuntimeData assembly_store_bundled_assemblies[];

// After: allocate at startup (size from config blob)
auto buffer = new uint8_t[config->total_uncompressed_size]();
auto assemblies = new AssemblyStoreSingleAssemblyRuntimeData[config->assembly_count]();
```

### 3. Environment variables → Java `Os.setenv()`

Generate Java code calling `Os.setenv()` before `initInternal()`, following the pattern NativeAOT already uses (`NativeAotEnvironmentVars.java`).

### 4. `pinvoke_preserve.*.ll` → Linker flags + `dlsym` (CoreCLR unified linking only)

This is the one file with actual executable code: `find_pinvoke()` maps `(library_hash, entrypoint_hash)` → function pointer via nested switch statements. It serves two purposes:

**Linker symbol preservation** — references to symbols like `@SystemNative_Bind` prevent `--gc-sections` from stripping them. Replace with `--undefined=` linker flags. `PinvokeScanner` already produces the symbol list, and `NativeLinker.cs` already supports `--export-dynamic-symbol` — the infrastructure is in place. (There's even a TODO in `dynamic.cc:88` where the team considered this approach.)

**Runtime P/Invoke resolution** — replace with `dlsym(RTLD_DEFAULT, entrypoint_name)`, which already exists as a fallback in `dynamic.cc`. P/Invoke results are cached by CoreCLR — each entrypoint is resolved once. Performance impact to be measured.

### 5. `jni_init_funcs.*.ll` → Generated C# (NativeAOT only)

Replace the LLVM IR function pointer array with generated C# using `[DllImport("__Internal")]`:

```csharp
static class JniInitFunctions
{
[DllImport("__Internal")]
static extern int JNI_OnLoad_SystemNative (IntPtr vm, IntPtr reserved);

[DllImport("__Internal")]
static extern int JNI_OnLoad_CryptoNative (IntPtr vm, IntPtr reserved);

public static void CallAll (IntPtr vm)
{
JNI_OnLoad_SystemNative (vm, IntPtr.Zero);
JNI_OnLoad_CryptoNative (vm, IntPtr.Zero);
}
}
```

NativeAOT compiles `[DllImport("__Internal")]` to direct native call instructions — zero overhead, compile-time symbol resolution, missing symbol = link error (not runtime crash). The `DirectPInvoke` infrastructure already exists in `Microsoft.Android.Sdk.NativeAOT.targets`.

## Performance

The primary goal is long-term maintainability and build simplification. However, this must not come at the expense of a significant measurable startup regression.

The proposed approach uses the same mmap-from-APK mechanism as today — config data is still accessed via direct pointer dereferences into memory-mapped regions. The main differences are: (1) eliminating `dlopen("libxamarin-app.so")` and its symbol resolution overhead, (2) replacing BSS pre-allocated buffers with dynamic `new[]`, and (3) replacing `find_pinvoke()` with `dlsym` for unified linking.

**We need to measure the actual performance impact** on real devices (high-end and low-end) with representative apps before and after. A feature flag should allow A/B comparison.

### Build time

| Current | Proposed |
|---|---|
| 5-7 `llc` invocations per ABI (LLVM IR compilation) | Eliminated |
| 1 `ld` invocation per ABI (native linking) | Eliminated |
| — | 1 `llvm-objcopy` per ABI (already used for assembly stores) |

## Work items

### Phase 1: Binary blob infrastructure
- [ ] `BinaryBlobWriter`: serialize `StructureInstance` to raw bytes using `StructureInfo` metadata
- [ ] Define blob header format (magic, version, section table)
- [ ] MSBuild task: reuse existing `LlvmIrComposer.Compose()` → `BinaryBlobWriter` → `DSOWrapperGenerator.WrapIt()`
- [ ] C++ `init_runtime_config()`: mmap blob from APK, parse header, set global pointers
- [ ] Convert `extern` declarations in `xamarin-app.hh` to static pointer globals (gated on MonoVM compat)

### Phase 2: Migrate data (incremental, per section)
- [ ] `ApplicationConfig` struct
- [ ] Runtime properties (name/value string tables for `coreclr_initialize()`)
- [ ] DSO cache + APK entries + name data
- [ ] Compressed assembly descriptors
- [ ] Pre-allocated buffers → dynamic `new[]`
- [ ] Environment variables → Java `Os.setenv()`
- [ ] JNI remapping tables

### Phase 3: Executable code replacements
- [ ] `pinvoke_preserve.ll` → `--undefined` linker flags + `dlsym(RTLD_DEFAULT)`
- [ ] `jni_init_funcs.ll` (NativeAOT) → generated C# with `[DllImport("__Internal")]`
- [ ] NativeAOT `environment.ll` → generated Java or C#

### Phase 4: Cleanup
- [ ] Remove `System.loadLibrary("xamarin-app")` for CoreCLR/NativeAOT
- [ ] Gate LLVM IR generators to MonoVM-only
- [ ] Remove `libxamarin-app.so` from CoreCLR/NativeAOT APK
- [ ] Gate `llc`/`ld` to MonoVM builds only

## Risks and mitigations

| Risk | Mitigation |
|---|---|
| Startup regression | Benchmark on real devices before/after. Feature flag for A/B. Old path remains until validated. |
| Struct layout drift (MSBuild writer vs C++ reader) | Reuse existing `StructureInfo` metadata for binary layout. Version header enables forward compat. |
| MonoVM compatibility | All changes gated behind runtime check. MonoVM path unchanged. |
| Desktop designer | `application_dso_stub.cc` remains. |

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.