dapr / dapr/components-contrib
Azure Service Bus Queue / Topic Subscription Filters for high-troughput scenarios
- Dominant language
- Go
- Stars
- 602
- Forks
- 580
- Avg merge
- 4d 9h
- Merged PRs (30d)
- 6
Description
## Describe the feature
Out of the box the Azure Service Bus topic subscription / queues support filters that are configurable on the Azure side. This way there could be filtering applied where the dapr sidecar doesn't get triggered which could be beneficial for high-throughput scenarios. This means messages are not propogated. (The filters could **only** be applied to message properties, and not message content!)
As Oli on the Discord channel described:
Basically Azure Service Bus Subscribers have a way to do filtering, which is essentially filtering at source. In high-throughput scenarios, this is desirable as it means messages won't needlessly propagate to the sidecar just to be dropped anyway.
https://learn.microsoft.com/en-us/azure/service-bus-messaging/service-bus-filter-examples
looks like the go SDK supports filters (or rules as I think they're called) . The filters would be set at the time of establishing the dapr Output channel for that particular subscriber -- I think i'm using the correct terminology here 😅
https://github.com/Azure/azure-sdk-for-go/blob/main/sdk/messaging/azservicebus/admin/admin_client_rules.go
@berndverst replied:
It wouldn't be an override but an addition I guess... so you would have to refrain from using the runtime filter feature.
We would not implement this by reusing the SDK filters that exist today because that is entirely intended for filtering in the runtime after the events are delivered to the sidecar (it is in memory filtering - which only sends the matching events to the app).
Instead, this would need to be exposed via subscription metadata -- the SDK and declarative subscriptions support metadata for subscriptions... so we'd have to agree on a way to specify it via that metadata, and then implement something in Service Bus which reads that metadata and initializes the right filter on the ServiceBus service side.
That would be a feature specific to ServiceBus. You can file a feature request for this - but 1.13 is the first release that could see this feature - and only if someone decides to work on it.
## Release Note
RELEASE NOTE: Service bus queue / topic subscription filtering for high troughput scenarios.
Contributor guide
Research direction
Start with the Service Bus component and its subscription metadata handling; no repository files or entry points are named in the issue. Read Azure's admin_client_rules.go and the linked filter examples first, then determine how subscription metadata could describe service-side rules and how completion would be verified without reusing runtime filters.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- azure, go
- Domain
- cloud, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100