[BUG/HELP] Cannot call host object, The parameter is incorrect. (0x80070057)

Description
I created a host object (I guess?) using some sample code around, but couldn't get it to work. I'm not an expert in COM stuff, but I don't think I'm doing anything out of this world.

I've also created a small custom wrapper so I can use NativeAOT with the DirectPInvoke and NativeLib properties. So far it's working but I'm having some issues with calling native objects.

C# side (caller). I've tried both classes (DomManipulationInterop and TestObject, both don't work):

 [ComVisible(true), InterfaceType(ComInterfaceType.InterfaceIsIDispatch)]
    public interface ITest
    {
        string SayHello(string message);
    }

    [ComVisible(true)]
    public class DomManipulationDotnetInterop : ITest
    {
        public string SayHello(string message) => message;
    }

    [Guid("B7A5C4C9-F4DA-4CD3-8D01-F7F42512ED08")]
    [ClassInterface(ClassInterfaceType.AutoDual)]
    [ComVisible(true)]
    [ProgId("TestHarnessApp.Test")]
    public class TestObject
    {
        public string Func(string param)
        {
            return "Func called with param : " + param.ToString();
        }
    }

DLLImport

[DllImport(DLLNAME)] public static extern int AddHostObject([MarshalAs(UnmanagedType.LPWStr)] string name, [MarshalAs(UnmanagedType.Interface), In] object obj);

C++ side:

// Assume wnd->comWindow exists. It's analogue to the webView2 interface.
DLLEXPORT HRESULT AddHostObject(LPCWSTR name, IDispatch* object)
{
	wil::unique_variant variant;
	wil::com_ptr<IDispatch> objectAsDispatch = object;
	objectAsDispatch.query_to<IDispatch>(&variant.pdispVal);
	variant.vt = VT_DISPATCH;
	HRESULT res = wnd->comWindow->AddHostObjectToScript(name, &variant);
	// variant.pdispVal->Release();
	return res;
}

Some notes:

• I can't pass obj as ref object obj because it'll throw an access violation reading memory
• I've tried with and without variant.pdispVal->Release();
• I've used the native debugger and I can confirm that the debugger can actually see the CLR object, I see it in the VS debugger.
• The function returns S_OK which means it went well. This is also confirmed by the fact that if I run something different in the WebView2 console, I get "Invalid number of parameters" instead of "The parameter is incorrect".
• I also tried using Marshal.GetIUnknownForObject or GetIDispatchForObject and changing the DllImport signature to sending an IntPtr instead, no workie.
• With above, changing the first two IDispatch to IUnknown, nothing either.

Version
SDK: 1.0.1072.54
Runtime: 97.0.1072.69
Framework: .NET 6 + C++
OS: Win11 ( Microsoft Windows [Version 10.0.22000.434] )

Any help is greatly appreciated, thanks!