ForNeVeR / ForNeVeR/Cesium

Universal processing of references and target frameworks

Open
#970 0 comments 0 reactions 0 assignees View on GitHub
area:compiler area:sdk kind:feature status:help-wanted
Dominant language
C#
Stars
451
Forks
49
Avg merge
13h 59m
Merged PRs (30d)
6

Description

Extracted from #947 and comments in #948.

## The Problem
When compiling C code, Cesium relies on the following two libraries:
- `Cesium.Runtime` of an appropriate/compatible version, containing Cesium runtime helper code and various types
- the "system library", meaning either `netstandard.dll` or `System.Runtime.dll`[^1]

[^1]: A fun fact is that while we technically support `SystemAssemblyKind.MsCorLib` to use `mscorlib.dll`, we don't really use that — when targeting .NET Framework, we rely on `netstandard.dll`. This seems to be a bug.

Cesium will add references to these libraries to the user application, and they are expected to be available to the user application in runtime. During compilation, Cesium will look up the required types from these libraries for the purpose of the compilation.

These two libraries are "hard-coded" in a sense — Cesium's `AssemblyContext` knows about them as `MscorlibAssembly` and `AssemblyContext.CesiumRuntimeAssembly`. If they are missing, compilation will break.

Cesium extracts the runtime library as `Cesium.Runtime.dll` from its own directory (so, uh, it will always be the netcore version, even under .NET Framework? another bug, probably) — and has a type `Cesium.CodeGen.TargetRuntimeDescriptor` to describe "the runtime". We have several runtimes hard-coded: we try to use .NET 6 library version for .NET 6+, .NET Standard 2.0 library for .NET Standard compilation, and (erroneously) `netstandard.dll` version 4.0 when targeting .NET Framework 4.8. This is a hard-coded set of frameworks; technically, Cesium only knows about these.

This situation has led us to several issues, including but not limited to #947. It is _hard_ to update Cesium to support new target frameworks currently.

## The Proposal
I propose a set of changes to how Cesium works with these libraries and project references in general, and also how the distinction between Cesium SDK or Cesium standalone compiler should work.

First, several postulates:
1. When compiling in the command line, it should be _easy_ (no additional arguments required or anything) to target _the current framework Cesium compiler runs on_. This should be the default option. Currently Cesium runs on .NET 10 — it should generate binaries and `.runtimeconfig.json` compatible with .NET 10.
2. It should be _possible_ to adjust this behavior to target any other version via the command-line compiler if needed. But some user actions (passing additional reference paths to `mscorlib.dll` or `netstandard.dll` or `Cesium.Runtime.dll` compatible with the current TFM) are acceptable. We shall support .NET Framework and .NET Standard as long as viable, while it's supported by the environment vendor.
3. Usage in a CMake toolchain or command line should be able to work without referencing NuGet packages or anything like that; this kind of usage should be _simple_ if targeting the "current framework".

Then, how we can achieve this.
1. Migrate away from the current explicit assembly paths (`AssemblyContext.MscorlibAssembly/CesiumRuntimeAssembly`, `TargetRuntimeDescriptor`) to a general concept of _reference assembly list_.
2. When resolving any necessary type (be it a type from `System` or from `Cesium.Runtime`), go through the list if pre-determined order and resolve the type. We should not hard-code which type comes from where, and go through a generic lookup process instead.
3. The Cesium SDK should pass the actual references from the framework (the set of reference assemblies determined by the SDK), and Cesium.Runtime obtained from a NuGet package to the list of references in the compiler by default. These should be first so that they have priority. I imagine the users might wish to customize these, so is should be possible to opt-out of this behavior .
4. The Cesium compiler (both the command-line one and CMake one) should allow passing one of these:
- a full list of reference assemblies (so, the user resolves `mscorlib.dll` and `Cesium.Runtime` themselves)
- a short standard framework name (`--framework net9.0/net10.0/net48/netstandard2.0`) — Cesium should be able to construct a correct set of references for these whenever possible
- mind that it may _not_ be possible if targeting a previous version of the current runtime — e.g. it might not be possible to target .NET 8 if Cesium runs on .NET 10; we should evaluate this more precisely and perhaps just forbid this kind of use
- a shorter target framework name (the current trinity `NetFramework/NetStandard/Net`) — allowing Cesium to choose the "best" version itself
- nothing (default): in this case, Cesium silently targets the "current runtime", .NET 10 currently

My understanding is that this is more or less whan Roslyn does (in its command-line form and SDK form), so this is what the ecosytem nudges us into.

## Implementation Notes
Quoting my text from https://github.com/ForNeVeR/Cesium/issues/947#issuecomment-3904840101:
> So, together with @pkazakov-dev we found how it works in C#.
>
> During build .NET SDK calculates `@(ReferencePathWithRefAssemblies)` (mostly calculated during `ResolveTargetingPackAssets` MSBuild target), and passed that to the C# compiler.
>
> The compiler then uses these assemblies as references/inputs for compilation.
>
> In fact, the same target is part of a compilation based on Cesium SDK as well! Cesium just ignores these items.
>
> We can _easily_ make Cesium to respect `@(ReferencePathWithRefAssemblies)` in either of two ways.
>
> 1. In Cesium SDK's compilation task, we could filter these input assemblies, locate the system runtime DLL among them (System.Runtime.dll, mscorlib.dll, netstandard.dll — which one depends on the target framework), and use it for further compilation.
>
> We could also include detection of Cesium.Runtime.dll here, although we might skip that for now, and focus on the system runtime assembly only.
> 2. Alternatively, we could change the way we approach the compilation. Right now, our two referenced assemblies (system runtime & Cesium runtime) are represented as two different hardcoded assemblies — two separate strict arguments.
>
> We could consider not treating them as two distinct assemblies, but instead implement a generic mechanism to search among referenced assemblies. All places where we currently look up things in the system runtime or in the Cesium runtime would need to be refactored to use a generalized search across references. Then this way of passing inputs to the compiler would work naturally on its own — no more distinct arguments, and just type resolve across the all referenced assemblies.

This issue is inspired by [this discussion thread](https://github.com/ForNeVeR/Cesium/pull/948#discussion_r2807592436), and is generally a follow-up to #948.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.