Allow contributions of tags of root worker metricsscope per workflow execution

We are using Temporal to handle workflows for registrations. Those registrations are for a certain region/country. We are sending the metrics to New Relic to create a dashboard to get insights in how many workflows have started, failed, cancelled or completed. This works fine. The region is part of the input that starts the workflow.

However we would like to split those metrics based on the region, we tried several hooks but apart from sending a custom metric on the start of a workflow we aren't able to solve this. What we want is that when a workflow starts, it contributes a tag to the metrics (in our case the region) so that we can use that as an additional dimension/facet in New Relic to group things by.

We are using the Java SDK and haven't found a foolproof way of doing that, apart from a custom metric at the start of the workflow.

What we currently have is we have a single WorkflowServiceStubs which we cannot use to set the region. We tried to create a WorkflowServiceStubs instance per region, but that starts to break as soon as we need to react to signals, we there loose the region and cannot use the proper one.

We tried to create a WorkflowClientInterceptor which doesn't give us access to the correct MetricScope to add tags.

We also tried a ContextPropagator although we can pass the region around there is no way to contribute it as a tag to the metrics.

All in all the only thing that worked was a custom metric at the start of a workflow, but that still leaves out the cancelled, failed and completed events we need to have.

It would be nice to have a way that when starting a workflow, we could add custom tags to the metrics and that those would be added to all events being send to New Relic. Currently it seems to hard or to be to inflexible.

Start by tracing metric-scope creation and propagation from WorkflowServiceStubs, WorkflowClientInterceptor, and ContextPropagator in the Java SDK. Determine how workflow-start metadata could reach metrics for completed, failed, and cancelled events; done means region-like tags are applied consistently to all requested workflow metrics without separate clients per region.