Feature request - Matcher continue on receiver failure

Setup

We have 2 Slack receivers: r1 and r2.

We have routes with matchers like so:

 - matchers:
	receiver: r2
	routes:
		- matchers:
		    - namespace!=""
              receiver: r1
         - matchers:
         	  receiver: r2

(sorry for formatting, can't get it right, you'll get the idea no?)

So what this does is:

• Have a main fallback to r1 in case nothing matches. However in this example, it should never hit that due to the last matcher always matching
• Have a matcher if namespace is not empty which routes to r1
• Have a second matcher which matches everything else (because continue=false by default) and goes to r2

Use-case

Now imagine having a more complicated setup with dynamic settings. For example dynamic Slack channels as such:

channel: some-channel-{{ (index .Alerts 0).Labels.namespace }}

The use-case here is to dynamically route stuff based on alert labels.
It's easy to check if a namespace is present, then route to a receiver with such channel variable. However the problem is that there is no way to know if the channel exists and if the alert gets send.

This would result in an error like:

level=error ts=2022-06-16T18:51:31.451Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=11 err="slack[0]: notify retry canceled due to unrecoverable error after 1 attempts: channel "****redacted****": unexpected status code 404: channel_not_found"

Feature request

 - matchers:
	receiver: r2
	routes:
		- matchers:
		    - namespace!=""
              receiver: r1
              continue_on_receiver_failure: true
         - matchers:
         	  receiver: r2

Having an option like continue_on_receiver_failure which would be the same as continue in its behaviour but only triggers when it hits an error while sending and then continues with other matchers.
The current log would not be a level=error anymore but a level=info and only goes into level=error if when no other matcher is going to catch/send it. For example keep the state and after the full evaluation of the routes, it should know if it did send it somewhere eventually.

Other info

I also think this is a very valuable thing to have in general. Let's say, some receiver endpoint like Slack is down. Then we can automatically fall back to an other receiver endpoint. Without having the need to constantly send it to both endpoints and introducing noise.

Start by tracing the receiver failure reported at dispatch.go:354 and the route evaluation behavior described in the issue. Review how matcher continuation and notification failures are currently handled, then determine how fallback delivery and logging should behave; done means the new option is documented, tested, and preserves existing behavior when unset.