Azure / Azure/azure-functions-dotnet-worker

Kafka Trigger: Failed to load the librdkafka native library on macOS ARM64 (Apple Silicon M1/M2/M3)

Open
#2,101 4 comments 2 reactions 1 assignee Claimed by @jainharsh98 View on GitHub
extensions: kafka needs-investigation
Dominant language
C#
Stars
466
Forks
215
Avg merge
3d 10h
Merged PRs (30d)
7

Description

When running an out-of-process isolated Azure Function v4 in .NET 6.0 using a Kafka Trigger (`Microsoft.Azure.Functions.Worker.Extensions.Kafka`) on macOS ARM64 (M1/M2/M3) the following error is encountered on startup:

```
[2023-11-28T15:23:44.201Z] A host error has occurred during startup operation '1f1ffdaf-dbc2-4c97-97a2-3bfc45e23ba0'.
[2023-11-28T15:23:44.202Z] Confluent.Kafka: Failed to load the librdkafka native library.
[2023-11-28T15:23:44.218Z] Failed to stop host instance '20345bf6-ed6a-476f-b069-334242f9eb8a'.
[2023-11-28T15:23:44.218Z] Microsoft.Azure.WebJobs.Host: The host has not yet started.
```

It looks like the native `librdkafka` binaries are not copied to the build output folder (i.e. `bin/Debug/net6.0`) on macOS ARM64.

To workaround this issue:
- Install `librdkafka` manually, for instance with Homebrew: `brew install librdkafka`
- Build the Azure Function Project
- Manually copy the following files in `/opt/homebrew/Cellar/librdkafka//lib` to the build output folder of the Function project (`bin/Debug/net6.0`):
- `librdkafka-static.a`
- `librdkafka.1.dylib`
- `librdkafka.a`
- `librdkafka.dylib`
- `librdkafka++.1.dylib`
- `librdkafka++.a`
- `librdkafka++.dylib`
- Run the Azure Function Project

Note that you have to follow the above steps again if you rebuild the Function project.

## See also
- https://github.com/Azure/azure-functions-kafka-extension/issues/425

## OS Information
- macOS 14.1.1 ARM64
- .NET 6.0
- Microsoft.Azure.Functions.Worker.Extensions.Kafka 3.10.0
- Azure Functions Version V4 (`dotnet-isolated`)
- Azure Functions Core Tools: 4.0.5455

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.