[3.0] Exanite's Tracking Issue

# Exanite's Tracking Issue

This tracks a bunch of miscellaneous tasks that I accumulated while working on Silk 3.
I'm open for anyone to work on these tasks, but please discuss with me in the Silk Discord first.

A task being here on this tracker does not mean it is guaranteed to be completed.
This tracker's primary goal is so that we do not accidentally forget about tasks and that any tasks not completed are intentionally left uncompleted.

## In-progress

Tasks in progress will be moved to the corresponding pull request.

## Uncategorized

This section contains informal reports, suggestions, or complaints about Silk 3.
I'm logging them here so they don't get forgotten about until I either open a formal issue or address them.

From Aqua:

- [ ] libSDL3.so is humongous (should not be nearly 14 MB)
- Exanite: I checked my system version and that one is only 3 MB.
- [ ] Silk 3 does not prefer its own native binaries? I.e., it prefers the system ones instead of the ones shipped by Silk.
- [ ] Go through the thread Aqua has made: https://discord.com/channels/521092042781229087/1512914692853465088

From Ethereal:

- [ ] SilkTouch generator takes an immense amount of RAM: https://discord.com/channels/521092042781229087/607634593201520651/1513592461493145601
- I need to also check with Curin to see how much RAM their device has. If their device doesn't have enough RAM, this might explain why the Microsoft job takes so long (aside from it already processing a lot of data).
- [ ] Run the Microsoft job locally and see how much RAM it takes on my device.

From Nolram (https://discord.com/channels/521092042781229087/607634593201520651/1524213012402147348):

- [ ] Add Imgui bindings
- Nolram recommended https://github.com/dearimgui/dear_bindings over cimgui (https://github.com/cimgui/cimgui)
- [ ] Investigate whether direct P/Invoke and static linking works in Silk 3 and how we want to support it.
- See: I'm not familiar with what direct P/Invoke means though
- Specifically, it involves some configuration, providing static builds, and maybe switching over to LibraryImport over DllImport.

From dfkeenan:

- [ ] Add SDL shadercross support: https://discord.com/channels/521092042781229087/587346162802229298/1543100984270520340
- My opinion on this: Shadercross is worth doing, worth considering the rest of the SDL addons as well, since Silk 3 leans much more heavily on SDL.
- [ ] "Trim the fat" from the generated bindings. Stuff like SDL's DUMMY_ENUM_VALUE should be removed
- See https://discord.com/channels/521092042781229087/587346162802229298/1545694628643340328

From me (Exanite):

- [ ] OpenCL `GetPlatformIDs`, `GetDeviceIDs`, and likely more functions do not have singular `GetPlatformID` equivalents. This should be handled by our function transformer so I'm guessing the XML metadata is missing. See https://github.com/dotnet/Silk.NET/pull/2590#discussion_r3741273750.
- [ ] Test projects should be added for all bindings. See https://github.com/dotnet/Silk.NET/pull/2590#discussion_r3836826213
- [ ] InlineArray structs should have implicit casts from strings/arrays, i.e., should behave like Ref. See https://github.com/dotnet/Silk.NET/pull/2590#discussion_r3836436842
- [ ] OpenXR types should be integrated with Silk.NET Maths
- See for interface approach (relevant for certain types where ABI is different): https://github.com/dotnet/Silk.NET/pull/2590#discussion_r3836444963
- See for remapping approach (relevant for types like vectors where ABI is the same): https://github.com/dotnet/Silk.NET/issues/506
- See for my initial list of relevant types: https://github.com/dotnet/Silk.NET/pull/2590#discussion_r3679747921
- [ ] API object devirtualization:
- See Perksey's comment and my comment here on potentially returning the concrete API class instead of the instance: https://github.com/dotnet/Silk.NET/pull/2590#discussion_r3741254562
- These comments were in response to my todo comments in the OpenCL example program
- Also see Perksey's idea to change the return something like `GL` where `T` is an `INativeContext` to remove one level of `callvirt`: https://discord.com/channels/521092042781229087/587346162802229298/1541090497185980478

From me (Exanite), as part of my OpenCL example program:

Original comments: https://github.com/dotnet/Silk.NET/blob/021738d5a4e9e77d777cf62f4337e522d6458df6/examples/CSharp/OpenCL/Tutorial001.SimpleKernel/Program.cs#L75-L80

- [ ] Expose INativeContext from the concrete API object where applicable (but not the interface)
- [ ] Discuss INativeContext API. Silk 3 is currently `LoadFunction(functionName, libraryNameHint)`, but I liked how Silk 2 only required the function name.

## High priority

Generator:

- [ ] `ExtractEnumConstants` does not preserve attributes placed on extracted constants (needs verification).

Bindings usage:

- [ ] Why don't we use the `InlineArrayN` types provided by .NET? Should we use them?
- Someone in the C# discord mentioned that these were added in .NET 10, which is what led to me add this task. This might be the reason we don't use them?
- [ ] OpenAL context should be disposable
- This is noted by Perksey in the OpenAL.Tutorial001.HelloSound project. I need to figure out what this Dispose method should do exactly.
- [ ] Check other API contexts as well for the same issue
channels/521092042781229087/1514735149915439274/1519099432442794150
- [ ] The magic `nullptr` constant does not work. It fails to resolve to anything.
- Apparently it works in the Silk 3 examples. Probably because it uses a ProjectReference instead of a PackageReference.
- [ ] The Vulkan bindings are double registering the LoaderInterface hook: https://discord.com/channels/521092042781229087/607634593201520651/1518220269519638599
- I'm going to implement a temporary hack fix, but I don't believe it to be the correct fix.
- We need a way to initialize an assembly consistently.
- Currently there are multiple entrypoints and they aren't guaranteed to be called.
- In Vulkan, we call `RegisterHook` in both Vk (defined in a manual file) and Vk.DllImport (defined in a generated file).
- This causes a multiple registration error, which admittedly is my fault since I didn't realize OpenAL didn't do the same pattern.
- Removing the duplicate call in Vk is the "fix" I mentioned above, but does not solve the underlying issue.
- Static constructors are only invoked if the static constructor's type or members are accessed.
- Notably, accessing Vk.DllImport does not trigger the static constructor for Vk, and vice versa.
- Because we assume/require both static constructors to be called for proper behavior (Vk.DllImport registers the hook and is not extensible, so we have to register alternative shared library names in Vk), having only one of them be called is a problem.
- [ ] Add option to configure which vendor suffixes `KhronosNonExclusiveVendor` can identify. Eg: For OpenGL, this should be only `ARB`.
- See conversation here: https://discord.com/channels/521092042781229087/587346162802229298/1531416150565453924
- [ ] Consider making function pointer structs work with `nullptr`. Seems like a nice QOL feature.
- [ ] OpenXR/CL/Vulkan: Add CurrentInstance/Device/Platform properties to interface partial.

Enums and other integral typedefs:

These have been proving to be consistently annoying to work with...
Vulkan and OpenXR have some special handling so this problem is less problematic, but still exists.

- [ ] I don't think the OpenGL bindings specify the method parameter types properly
- Eg: `void DrawElements(uint mode, uint count, uint type, void* indices);`
- This should use the enum type instead of `uint`.
- [ ] Vulkan: Some types are incorrect. `uint` is used instead of `MaybeBool`. https://discord.com/
- [ ] OpenCL: Same issue. Tracked in the smoke test project as todos.

Documentation:

- [ ] Add Vulkan smoke test project
- [ ] Differentiate between smoke tests and actual tutorials/examples. I would categorize the current SDL/OpenAL "tutorials" in Silk 3 to be smoke tests since they really shouldn't be referenced as sources of best practices.
- [ ] Add bindings usage docs
- [ ] Explain that the native docs for each API can be used since Silk doesn't really change how you use the API
- [ ] Explain difference between bindings and high level APIs
- Ie, high level APIs do not represent bindings so usually the native docs don't apply for those
- [ ] Explain the different DSL types and magical constants (eg: `nullptr`)

Maintenance:

- [ ] Consider removing `ModCSharpSyntaxRewriter`
- [ ] Add SilkConstants class to generator
- Ideally all magic numbers and constants are consolidated here and documented.
- [ ] Add `` comment to start of each generated file. This has the effect of disabling IDE/linter/etc checks, which means less noise and performance cost of working in the Silk repo.

DI:

- [ ] Fix issues when running multiple jobs
- [ ] Figure out why the headers are getting mixed up when running multiple jobs at the same time.
- Seems to be related to `TransformFunctions`.
- [ ] Figure out why the SDL handle structs are getting named as `Silk.NET.SDL.ConditionHandle.gen.cs` instead of `ConditionHandle.gen.cs` when running multiple jobs at the same time.
- [ ] Maybe rework how using statements are added. Not sure why, but using statements are randomly added/removed between Windows/Linux. This causes noisy diffs.
- [ ] Investigate how DI is set up in this project. I believe it is the core cause of the multi-job isolation issues. I need to verify this, but the jobs seem to share the same mod instances instead of being separated. This would explain why we need to access data by job key instead of it being simply injected into the mod instances.
- [ ] Rework registration/configuration system to be in pure C# instead of JSON
- This also is related to reworking the DI setup.
- This is something I volunteered to do.
- See: https://discord.com/channels/521092042781229087/587346162802229298/1530907649561067693

## Medium priority

Bindings:

- [ ] Use `CLong` where possible in the bindings.
- [ ] For enums, use `Constant`, and theoretically it should work.
- See conversation here: https://github.com/dotnet/Silk.NET/pull/2590#discussion_r3699967777
- [ ] Consider implementing a `IsProbablyABitmask` function based on [BuildTool's IsProbablyABitmask](https://github.com/dotnet/Silk.NET/blob/c87fc664cc5d15abd754997b0a9ad138470554c4/src/Core/Silk.NET.BuildTools/Common/Utilities.cs#L627) for identifying flags enums
- Original comment: https://github.com/dotnet/Silk.NET/pull/2503#discussion_r2662888928

Quality:

- [ ] Add some way of verifying our bindings are well polished.
- These would mainly involve simple automated sanity checks that ensure our bindings our consistent. Could be done as a separate CLI project from SilkTouch.
- [ ] Check for missing namespaces.
- This was an issue discovered during https://github.com/dotnet/Silk.NET/pull/2587. I've fixed one case of these, but haven't verified the rest of the generated files (since there's way too many to manually verify).
- [ ] Check for spelling/casing issues? Identifiers like `Sdl.PropGpuDeviceCreateDebugmodeBoolean` are technically correct (native name is `SDL_PROP_GPU_DEVICE_CREATE_DEBUGMODE_BOOLEAN`, so `Debugmode` is "one" word), but we could have an audit tool that checks for these non-ideal cases so we can add manual overrides. Worth the effort? Probably not unless we care about perfecting the bindings.

Maintenance:

- [ ] ClangSharp switched from System.CommandLine to a custom command line argument parser. I noticed that we had verbatim copied code from ClangSharp for handling arguments in Silk 3, so we should update that.
- [ ] This also lets us update System.CommandLine, which if I recall, we are still using a beta version of.

## Low priority

CI:

- [ ] Prevent native builds from triggering unnecessarily when the PR description is edited
- Native builds should only trigger once. If the last commit in the PR is from the native builds workflow, it should not run again.
- [ ] Figure out how to make our CI workflows work with forks.
- I did some testing here and got a proof-of-concept working: https://github.com/Exanite-Dev/GithubActionTesting
- Note that we need to be careful about security here. This involves escalation of privileges.

Maintenance:

- [ ] Cross reference changelog and past issues for Silk 2 to ensure that we don't have the same bugs in Silk 3

High level utilities:

- [ ] Reimplement the Vulkan struct chaining API
- [ ] Add image format utils for Vulkan/etc

IDE / Metadata:

- [ ] Ensure SupportedApiProfiles attributes are correct
- [ ] Properly resolve API profiles for Flags/FlagBits types
- [ ] Prefer name from `NativeName` for resolving API profiles
- [ ] Consider optimizing `NativeName` attributes.
- This should involve renaming `StripAttributes` to `CleanupAttributes` and adding an option in its config to optimize `NativeName` attributes.
- Also see: https://github.com/dotnet/Silk.NET/pull/2503#discussion_r2662921224

## Completed

Completed during https://github.com/dotnet/Silk.NET/pull/2588:

- [x] Switch to SLNX solution format to improve readability/mergability
- [x] From Ethereal: They had a CSharpier TypeLoadException when running the Win32 job on `develop/3.0`. Might be related to the error I saw half a year ago. If so, likely can be fixed by simply updating CSharpier. CSharpier has updated their Roslyn package since then.

Completed during https://github.com/dotnet/Silk.NET/pull/2591:

- [x] Port back the `--only` option from Curin's branch so I don't need to specify `--skip` for all but one job

Completed or planned to be completed during https://github.com/dotnet/Silk.NET/pull/2590 (not yet merged):

- [x] Begin work on the rest of the Khronos bindings
- I'd like to start this early because this lets us spot issues early and we can use these as test cases for the rest of the changes.
- [x] Generator usage: Explain SilkTouch CLI options
- [x] Generator usage: Clarify that SilkTouch does not modify non `.gen.cs` `.cs` files during the `Generated Bindings Output` section
- [x] Consider reworking data type trimming regexes.
- This was discussed more here: https://github.com/dotnet/Silk.NET/pull/2503#issuecomment-3622907776
- The summary is that the current set of 2 regexes doesn't elegantly handle the case where the ending of a word should not be trimmed, but the word itself can have data type suffixes.
- For example: `alSourceRewindv` and `alSourceRewind`. The `d` in `Rewind` can be mistaken as a data type suffix, so we add it to the `EndingsNotToTrim` regex. However, the `dv` in `Rewindv` also has the same problem.
- We likely need something more akin to `WordsNotToTrimInto` than `EndingsNotToTrim`.
- [x] Investigate why we currently use DllImport in Silk 3.
- Also added some light documentation to `AddVTables` in the mod MD docs.

Completed in https://github.com/dotnet/ClangSharp/pull/699

- [x] Consider opening a ClangSharp PR to fix issue with `SDL_MAX_SINT64` generation on Linux
- More context: https://discord.com/channels/521092042781229087/1376331581198827520/1443401810822828113
- `public const nint SDL_MAX_SINT64 = unchecked(0x7FFFFFFFFFFFFFFF);` does not compile

Completed outside of a PR:

- [x] For `DrawElements`/etc, check if the pointer to 0 issue present in Silk 2 still exists in Silk 3
- [x] Investigate using the new BitCast optimizations: https://discord.com/channels/143867839282020352/312132327348240384/1517589517727436952
- Eg: BitCasting a user struct to a Vector struct and immediately using an SIMD operation is now properly optimized. Free performance!
- This mainly affects Otac0n's work, but I'm logging it here so I also keep it in mind as I work on the bindings.
- Decision: Just going to keep this in mind. Don't think I need to actively review the bindings for this since there's not many places where this will be used.

## Not doing

For transparency and for reference, I'll move any tasks I don't plan on doing here.

- ~~Figure out how to handle CLA properly. Currently native builds will block CLA due to the use of a bot account.~~
- Seems to be fine now.