dapr / dapr/components-contrib

Azure Service Bus Queue / Topic Subscription Filters for high-troughput scenarios

Open
#3,078 0 comments 0 reactions 0 assignees View on GitHub
good first issue help wanted kind/enhancement
Dominant language
Go
Stars
602
Forks
580
Avg merge
4d 9h
Merged PRs (30d)
6

Description

## Describe the feature
Out of the box the Azure Service Bus topic subscription / queues support filters that are configurable on the Azure side. This way there could be filtering applied where the dapr sidecar doesn't get triggered which could be beneficial for high-throughput scenarios. This means messages are not propogated. (The filters could **only** be applied to message properties, and not message content!)

As Oli on the Discord channel described:
Basically Azure Service Bus Subscribers have a way to do filtering, which is essentially filtering at source. In high-throughput scenarios, this is desirable as it means messages won't needlessly propagate to the sidecar just to be dropped anyway.

https://learn.microsoft.com/en-us/azure/service-bus-messaging/service-bus-filter-examples

looks like the go SDK supports filters (or rules as I think they're called) . The filters would be set at the time of establishing the dapr Output channel for that particular subscriber -- I think i'm using the correct terminology here 😅

https://github.com/Azure/azure-sdk-for-go/blob/main/sdk/messaging/azservicebus/admin/admin_client_rules.go

@berndverst replied:
It wouldn't be an override but an addition I guess... so you would have to refrain from using the runtime filter feature.

We would not implement this by reusing the SDK filters that exist today because that is entirely intended for filtering in the runtime after the events are delivered to the sidecar (it is in memory filtering - which only sends the matching events to the app).

Instead, this would need to be exposed via subscription metadata -- the SDK and declarative subscriptions support metadata for subscriptions... so we'd have to agree on a way to specify it via that metadata, and then implement something in Service Bus which reads that metadata and initializes the right filter on the ServiceBus service side.

That would be a feature specific to ServiceBus. You can file a feature request for this - but 1.13 is the first release that could see this feature - and only if someone decides to work on it.

## Release Note
RELEASE NOTE: Service bus queue / topic subscription filtering for high troughput scenarios.

Contributor guide

Open the contributing guide

Research direction

Start with the Service Bus component and its subscription metadata handling; no repository files or entry points are named in the issue. Read Azure's admin_client_rules.go and the linked filter examples first, then determine how subscription metadata could describe service-side rules and how completion would be verified without reusing runtime filters.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure, go
Domain
cloud, distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.