Invoking an unmanaged fastcall function pointer under x86 Windows fails at runtime

Open
#113,851 1 comment 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
35/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Stale
Tech stack
csharp

Research direction

Start with the Windows x86 calling-convention handling in src/coreclr/vm/dllimport.cpp around line 4238. Reproduce the TypeLoadException using the provided unsafe C# function-pointer example and inspect how Fastcall is handled. Done means the UnmanagedCallersOnly Fastcall pointer invocation works on Windows x86 without the unsupported-calling-convention failure.

Written by the indexing model from the issue text.

Description

arch-x86 area-Interop-coreclr in-pr
Description

Exporting a function using UnmanagedCallersOnly and the Fastcall calling convention will work, but calling ((delegate* unmanaged[Fastcall]<int, int>)somePtr)(someArg) will fail with

System.TypeLoadException
  HResult=0x80131522
  Message=Unsupported unmanaged calling convention.
Reproduction Steps
unsafe
{
    int x = ((delegate* unmanaged[Fastcall]<int, int>)IntPtr.Zero)(0);
}
Expected behavior

The function pointer should be called.

Actual behavior

The call fails due to the Fastcall calling convention being unimplemented.

Regression?

Unlikely that this was implemented before and removed afterwards.

Known Workarounds

No response

Configuration

Windows x86

Technically I'm running from a CoreCLR embedded in an application, but it should not matter in this scenario.

Other information

https://github.com/dotnet/runtime/blob/f17c21c1d1d4dd8825bfc378858b5fdb9375c6a5/src/coreclr/vm/dllimport.cpp#L4238

Dominant language
C#
Stars
18.3k
Forks
5.6k
PR merge metrics
PR metrics pending

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from dotnet/runtime

All issues in dotnet/runtime

Similar issues

More C# issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.