Azure / Azure/azure-functions-dotnet-worker

[Feedback] Custom trigger and isolated worker mode

Open
#1,177 11 comments 4 reactions 1 assignee Claimed by @kshyju View on GitHub
area: binding question
Dominant language
C#
Stars
466
Forks
215
Avg merge
3d 10h
Merged PRs (30d)
7

Description

Hi,

Following .NET 7 release, which included removal of support for in-process function execution model, I'd like to share how I tried to adhere to the isolated worker approach for my custom trigger, and issues we are facing.

I'm doing it in Issues as Dicussions space is not open.

## Context

### MassTransit
In my company, we are heavy users of the [MassTransit](https://github.com/MassTransit/MassTransit) ([doc site](https://masstransit-project.com/)) library. We have plenty of procedures and ways of doing that are linked to how MassTransit is behaving.

One of them is the automatic topics, subscriptions, and queues creation at runtime, from message types, based on conventions (yes, it required credentials allowed to do so).
Others include
* how errors are managed, through creation of `_errors` (something went bad during message consumption) and `_skipped` (no consumer for the message type was configured) queues.
* the amount of diagnostics provided directly on the message
* and the ability to configure/extend the behavior of the lib through a very rich code approach

![image](https://user-images.githubusercontent.com/8553578/201680807-77e2b5f7-9816-4fe7-9cbc-90980a52dcde.png)

### Custom trigger
We previously used WebJobs a lot to consume messages, but we decided to move to functions.

To that end, we created a custom trigger that would allow us to get all the benefits from MassTransit library with the functions runtime.

You can find a very initial version [here](https://github.com/YounitedCredit/Younited.MassTransit.Trigger), that we published as part of a blob post, which should be simple enough to understand how it works.

If you take a look at the [README](https://github.com/YounitedCredit/Younited.MassTransit.Trigger/blob/master/README.md), you will see that we relied on dependency injection to inject code configuration for the MassTransit library.

The result is quite crude, but works perfectly.

## Issues encountered migrating to Isolated Worker

### ExtensionInformation relying on nuget
The fact that the `ExtensionInformationAttribute` is relying on a nuget package is painful during development, and requires specific setup to be able to debug something.
This is not a big issue, but it does not help

### Ability to inject into the host
As the plugin relies on code to configure the trigger, we had to find a way to inject code into the host.
The best solution we could find was to code an extension, as the extension would be executed in the host.
As the configuration code relies on message types, we had to ensure the message types would be in a dedicated project too (not a big issue in our case, it was already the case to distribute them to clients through a specific nuget).
Of course, for the trigger users, it would be more painful, as they would have to package a project as an extension, but at least it would work.

We did not find how to inject into the build process to register an extension, but it seems hard to do. So instead, we relied on manual modifications of `extensions.json`, and manual copy of files into the `.azurefunctions`. We had to add a few System.* dll for .NET 7, but this is probably to be expected as we hacked the build process.

So possible, but hard to find how to automate the extension registration in a simple way.

### Unavailability of the parameter type on the trigger extension.
As said above, MassTransit uses conventions based on message types to configure the infrastructure.
Even having the configuration extension running in the host, we discovered that the GRPC protocol did not transfer the attributed parameter type from the worker to the host, presenting it as a simple string, while the trigger [expects it to be set to a proper parameter type](https://github.com/YounitedCredit/Younited.MassTransit.Trigger/blob/cd523e1cab96426e5af328a99e68007f000a5038/src/Younited.MassTransit.Trigger/Binding/MassTransitTriggerBindingProvider.cs#L33-L50).

As such, even relying on previous hacks, this made clear we would not be able to implement the customer trigger we wanted.

## Conclusion

Could you clarify exactly where you are headed towards feature parity for custom triggers, and maybe a little bit of a timeframe for this parity to occur ?

The removal of the in-process model broke our ability to rely on functions, with little alternatives left currently, in a (probably small with regards to the functions world, but still) range of use cases.

Could we expect being able to inject configuration into the host easily ? And to have access to (nearly) the same information from an isolated worker than from an in-process work.

Thanks for reading !

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.