[NativeAOT] Passing an RCW COM interface to a P/Invoke method crashes in NativeAOT
- Dominant language
- C#
- Stars
- 18.3k
- Forks
- 5.6k
- PR merge metrics
- PR metrics pending
Description
### Description
I'm trying to pass an RCW to a P/Invoke method. In normal compiled .Net, the object is passed correctly as the original native pointer, and everything works correctly. In NativeAOT the program crashes when trying to call a P/Invoke method with an `IUnknown` object `in` parameter.
### Reproduction Steps
```csharp
[ComImport]
[InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
[Guid("C859BA31-821B-4947-A04F-5C30C6A94B82")]
public interface IFoo
{
[PreserveSig]
int Bar([In][MarshalAs(UnmanagedType.LPStr)] string Message);
}
public class FooComWrapper : IFoo
{
public struct IFooVtbl
{
public IntPtr Bar;
}
public IntPtr Instance { get; }
public IFooVtbl Vtbl { get; }
public delegate int BarDelegate(IntPtr ptrThis, [MarshalAs(UnmanagedType.LPStr)] string Message);
public BarDelegate BarFunction { get; set; }
public FooComWrapper(IntPtr ptr)
{
Instance = ptr;
unsafe
{
Vtbl = Marshal.PtrToStructure(new nint(*(void**)ptr.ToPointer()));
BarFunction = Marshal.GetDelegateForFunctionPointer(Vtbl.Bar);
}
}
public int Bar(string Message)
{
return BarFunction(Instance, Message);
}
}
public class FooComWrappers : ComWrappers
{
protected override unsafe ComInterfaceEntry* ComputeVtables(object obj, CreateComInterfaceFlags flags, out int count)
{
ComInterfaceEntry* entry = null;
count = 0;
if (obj is FooComWrapper foo)
{
count = 1;
entry = (ComInterfaceEntry*)RuntimeHelpers.AllocateTypeAssociatedMemory(typeof(FooComWrappers), sizeof(ComInterfaceEntry));
entry->Vtable = *(IntPtr*)foo.Instance.ToPointer();
entry->IID = typeof(IFoo).GUID;
}
return entry;
}
protected override object? CreateObject(nint externalComObject, CreateObjectFlags flags)
{
return new FooComWrapper(externalComObject);
}
protected override void ReleaseObjects(IEnumerable objects)
{
throw new NotImplementedException();
}
}
public static class Program
{
[DllImport("CppProject")] static extern int FooFactory(out IntPtr ptr);
[DllImport("CppProject")] static extern int FooFunction(IFoo foo);
[SupportedOSPlatform("Windows")]
public static void Main(string[] args)
{
ComWrappers.RegisterForMarshalling(new FooComWrappers());
if (FooFactory(out IntPtr ptrFoo) == 0)
{
if (Marshal.GetObjectForIUnknown(ptrFoo) is IFoo foo)
{
foo.Bar("RCW"); // Works
_ = FooFunction(foo); // <<<<<< Crashes in NativeAOT on the aforementioned issue
}
}
}
}
```
### Expected behavior
I expect the object to be passed in to the `FooFunction` as the unwrapped original pointer. This is the behavior I observe in normal .Net compilation.
### Actual behavior
In NativeAOT compilation, the object is first cast to a `ManagedObjectWrapper` pointer and then `QueryInterface` is called using on the `ManagedObjectWrapper`. This will AV because the object is not of type ManagedObjectWrapper. Code in question here: https://github.com/dotnet/runtime/blob/2fc23e68a49aeea2458d4a0e20fc4cc232930ff3/src/libraries/System.Private.CoreLib/src/System/Runtime/InteropServices/ComWrappers.cs#L1564
I believe there is a bug here and if QueryInterface needs to be called then we should cast to an IUnknown pointer. I am also confused with normal .Net compilation doesn't seem to take this path?
### Regression?
_No response_
### Known Workarounds
We can use `Marshal.QueryInterface` to get the original pointer and change the P/Invoke signature to accept the pointer and pass that pointer to the function. This will however not work when using external library code because we would be unable to change argument types.
### Configuration
.Net 10
### Other information
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.