microsoft / microsoft/aspire

AddKafkaConsumer wrong health check configuration

Open
#5,904 2 comments 0 reactions 0 assignees View on GitHub
area-integrations kafka
Dominant language
C#
Stars
6.3k
Forks
991
Avg merge
2d 15h
Merged PRs (30d)
196

Description

### Is there an existing issue for this?

- [X] I have searched the existing issues

### Describe the bug

When `AddKafkaConsumer` adds health check options it sets them via a `ProducerConfig` as opposed to a `ConsumerConfig`:

```csharp
builder.TryAddHealthCheck(new HealthCheckRegistration(healthCheckName,
sp =>
{
var connectionFactory = serviceKey is null
? sp.GetRequiredService>()
: sp.GetRequiredKeyedService>(serviceKey);

var options = new KafkaHealthCheckOptions();
options.Configuration = new ProducerConfig(connectionFactory.Config.ToDictionary());
options.Configuration.SocketTimeoutMs = 1000;
options.Configuration.MessageTimeoutMs = 1000;
options.Configuration.StatisticsIntervalMs = 0;
return new KafkaHealthCheck(options);
},
failureStatus: default,
tags: default));
```

Source [here](https://github.com/dotnet/aspire/blob/56b44c55591d9f2d66a2616b3f27517dcedf08bd/src/Components/Aspire.Confluent.Kafka/AspireKafkaConsumerExtensions.cs#L147-L152).

However when you setup your Kafka integration you may use properties only meant for consumers:

```csharp
builder.AddKafkaConsumer("eventStreamer", configure =>
{
configure.Config.GroupId = "basket";
configure.Config.AutoOffsetReset = AutoOffsetReset.Earliest;
configure.Config.EnableAutoCommit = false;
});
```

Because of this you get an endless list of Kafka warnings every time the readiness health probe runs:

```
%4|1727213776.286|CONFWARN|rdkafka#producer-107| [thrd:app]: Configuration property group.id is a consumer property and will be ignored by this producer instance
%4|1727213776.286|CONFWARN|rdkafka#producer-107| [thrd:app]: Configuration property enable.auto.commit is a consumer property and will be ignored by this producer instance
%4|1727213776.286|CONFWARN|rdkafka#producer-107| [thrd:app]: Configuration property auto.offset.reset is a consumer property and will be ignored by this producer instance
```

### Expected Behavior

Health checks for Kafka consumers should be configured via a `ConsumerConfig`, not a `ProducerConfig`.

### Steps To Reproduce

1. Ensure your app is calling `AddServiceDefaults` and `MapDefaultEndpoints`

2. Add Kafka integration to your app:
```
dotnet add package Aspire.Confluent.Kafka
```

3. Add your consumer, specifying GroupId, AutoOffsetReset and EnableAutoCommit:

```csharp
builder.AddKafkaConsumer("eventStreamer", configure =>
{
configure.Config.GroupId = "basket";
configure.Config.AutoOffsetReset = AutoOffsetReset.Earliest;
configure.Config.EnableAutoCommit = false;
});
```

4. Start your Kafka container

5. Run the app

7. Ping the Ready endpoint:
```http
GET http://localhost:5287/health/ready
```

8. Observe the warnings in your console:
```
%4|1727214415.021|CONFWARN|rdkafka#producer-2| [thrd:app]: Configuration property group.id is a consumer property and will be ignored by this producer instance
%4|1727214415.021|CONFWARN|rdkafka#producer-2| [thrd:app]: Configuration property enable.auto.commit is a consumer property and will be ignored by this producer instance
%4|1727214415.022|CONFWARN|rdkafka#producer-2| [thrd:app]: Configuration property auto.offset.reset is a consumer property and will be ignored by this producer instance
```

### Exceptions (if any)

_No response_

### .NET Version info

.NET SDK:
Version: 8.0.400
Commit: 36fe6dda56
Workload version: 8.0.400-manifests.71228b2b
MSBuild version: 17.11.3+0c8610977

Runtime Environment:
OS Name: Windows
OS Version: 10.0.22631
OS Platform: Windows
RID: win-x64
Base Path: C:\Program Files\dotnet\sdk\8.0.400\

.NET workloads installed:
Configured to use loose manifests when installing new manifests.
[aspire]
Installation Source: SDK 8.0.400, VS 17.11.35303.130
Manifest Version: 8.2.0/8.0.100
Manifest Path: C:\Program Files\dotnet\sdk-manifests\8.0.100\microsoft.net.sdk.aspire\8.2.0\WorkloadManifest.json
Install Type: Msi

Host:
Version: 8.0.8
Architecture: x64
Commit: 08338fcaa5

.NET SDKs installed:
8.0.108 [C:\Program Files\dotnet\sdk]
8.0.206 [C:\Program Files\dotnet\sdk]
8.0.304 [C:\Program Files\dotnet\sdk]
8.0.400 [C:\Program Files\dotnet\sdk]

.NET runtimes installed:
Microsoft.AspNetCore.App 8.0.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 8.0.8 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.NETCore.App 8.0.6 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 8.0.8 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.WindowsDesktop.App 8.0.6 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 8.0.8 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

Other architectures found:
x86 [C:\Program Files (x86)\dotnet]
registered at [HKLM\SOFTWARE\dotnet\Setup\InstalledVersions\x86\InstallLocation]

Environment variables:
Not set

global.json file:
Not found

Learn more:
https://aka.ms/dotnet/info

Download .NET:
https://aka.ms/dotnet/download

### Anything else?

IDE: VS Code

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.