dapr / dapr/dotnet-sdk

dead letter subscription not working with pub sub routing

Open
#998 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
C#
Stars
1.2k
Forks
378
Avg merge
1d 22h
Merged PRs (30d)
5

Description

I use the following declarative subscription.yaml for pub sub routing
```
apiVersion: dapr.io/v2alpha1
kind: Subscription
metadata:
name: mysubscriptions
namespace: default
spec:
pubsubname: pubsub
topic: inventory
deadLetterTopic: poisonMessages
routes:
rules:
- match: event.type == "Widget"
path: /inventory/widgets
- match: event.type == "Gadget"
path: /inventory/gadgets
default: /inventory/products
scopes:
- pubsubrouting-service
```

and the following endpoints
```
app.MapPost("/inventory/widgets", (Widget widget) =>
{
Console.WriteLine("Subscriber received : " + widget);
return Results.Ok();
});

app.MapPost("/inventory/gadgets", (Gadget gadget) =>
{
Console.WriteLine("Subscriber received : " + gadget);
return Results.Ok();
});

app.MapPost("/inventory/products", (Product product) =>
{
Console.WriteLine("Subscriber received (default) : " + product);
return Results.Ok();
});

app.MapPost("/failedMessages", (JsonDocument message) =>
{
Console.WriteLine("Subscriber received (dead letter) : " + message);
return Results.Ok();
});
```

The routing works fine for a Widget, Gadget and also for a Thingamajig. Widget, Gadget and Thingamajig all derive from a base abstract Product. Thingamajig is correctly consumed as a Product by the endpoint for the default route thanks to polymorphic deserialization in STJ in .NET 7.0.

If a NotAProduct is published to the inventory topic then this cannot be consumed by the endpoint for the default route and the message is published to the dead letter topic.

I also have a deadletter-subscription.yaml that subscribes to the dead letter topic.
```
apiVersion: dapr.io/v2alpha1
kind: Subscription
metadata:
name: deadlettertopics
namespace: default
spec:
pubsubname: pubsub
topic: poisonMessages
route: /failedMessages
scopes:
- pubsubrouting-service
```

However, this dead letter subscription fails to deliver the message to the endpoint. In the logs I see
```
level=warning msg="retriable error returned from app while processing pub/sub event , topic: inventory, body: System.NotSupportedException: Deserialization of types without a parameterless constructor, a singular parameterized constructor, or a parameterized constructor annotated with 'JsonConstructorAttribute' is not supported. Type 'PubSubRouting.Interfaces.Product'. Path: $ | LineNumber: 0 | BytePositionInLine: 15.\n ---> System.NotSupportedException: Deserialization of types without a parameterless constructor, a singular parameterized constructor, or a parameterized constructor annotated with 'JsonConstructorAttribute' is not supported. Type 'PubSubRouting.Interfaces.Product'.\n --- End of inner exception stack trace ---\n at System.Text.Json.ThrowHelper.ThrowNotSupportedException(ReadStack& state, Utf8JsonReader& reader, NotSupportedException ex)\n at System.Text.Json.Serialization.Converters.ObjectDefaultConverter`1.OnTryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)\n at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)\n at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)\n at System.Text.Json.Serialization.JsonConverter`1.ReadCoreAsObject(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)\n at System.Text.Json.JsonSerializer.ReadCore[TValue](Utf8JsonReader& reader, JsonTypeInfo jsonTypeInfo, ReadStack& state)\n at System.Text.Json.JsonSerializer.ContinueDeserialize[TValue](ReadBufferState& bufferState, JsonReaderState& jsonReaderState, ReadStack& readStack, JsonTypeInfo jsonTypeInfo)\n at System.Text.Json.JsonSerializer.ReadFromStreamAsync[TValue](Stream utf8Json, JsonTypeInfo jsonTypeInfo, CancellationToken cancellationToken)\n at Microsoft.AspNetCore.Http.HttpRequestJsonExtensions.ReadFromJsonAsync(HttpRequest request, Type type, JsonSerializerOptions options, CancellationToken cancellationToken)\n at Microsoft.AspNetCore.Http.HttpRequestJsonExtensions.ReadFromJsonAsync(HttpRequest request, Type type, JsonSerializerOptions options, CancellationToken cancellationToken)\n at Microsoft.AspNetCore.Http.RequestDelegateFactory.g__TryReadBodyAsync|89_0(HttpContext httpContext, Type bodyType, String parameterTypeName, String parameterName, Boolean allowEmptyRequestBody, Boolean throwOnBadRequest)\n at Microsoft.AspNetCore.Http.RequestDelegateFactory.<>c__DisplayClass89_2.<b__2>d.MoveNext()\n--- End of stack trace from previous location ---\n at Microsoft.AspNetCore.Routing.EndpointMiddleware.g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)\n at Dapr.CloudEventsMiddleware.ProcessBodyAsync(HttpContext httpContext, String charSet)\n at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddlewareImpl.Invoke(HttpContext context)\n\nHEADERS\n=======\nHost: 127.0.0.1:80\nUser-Agent: fasthttp\nContent-Type: application/cloudevents+json\ntraceparent: 00-f40eea4a4537a1c0c5060fc62864a079-f063c2e3735d44d9-01\nContent-Length: 398\nPubsubname: pubsub\n. status code returned: 500" app_id=pubsubrouting-service instance=cea8108eec8c scope=dapr.runtime type=log ver=1.9.5
level=debug msg="rabbitmq pub/sub: publishing message to poisonMessages" app_id=pubsubrouting-service instance=cea8108eec8c scope=dapr.contrib type=log ver=1.9.5
level=debug msg="rabbitmq pub/sub: acking message '' from topic 'inventory'" app_id=pubsubrouting-service instance=cea8108eec8c scope=dapr.contrib type=log ver=1.9.5
level=debug msg="no matching route for event in pubsub pubsub and topic poisonMessages; skipping" app_id=pubsubrouting-service instance=cea8108eec8c scope=dapr.runtime type=log ver=1.9.5
level=debug msg="rabbitmq pub/sub: acking message '' from topic 'poisonMessages'" app_id=pubsubrouting-service instance=cea8108eec8c scope=dapr.contrib type=log ver=1.9.5
```

The logs say there is no matching route for the event in the deadletter subscription and the message is simply acknowledged. What am I missing? Please advise

You can find a repro here:
https://github.com/RemcoBlok/dapr-samples/tree/main/pub-sub-routing-http

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.