Rule severity vs diagnostic severity

From https://github.com/PowerShell/PSScriptAnalyzer/issues/1610.

One consideration that comes up when authoring rules is:

• Rules can have a "severity"
• The individual diagnostics can also have a "severity"

When we configure things, this creates the issue of:

• Does a configured severity refer to a filter on rules or diagnostics?
• If it's rules, how do rules perform broad analysis on a topic area and possibly output diagnostics of different severity?
  • Should rules designate their severity as the highest severity they may emit? If so, how do we enforce that statically?
• If it's diagnostics, how do we ensure rules are run efficiently? If we can't statically determine ahead of time which rules will emit diagnostics of the right severity, are we forced to always run all the rules?

We could potentially offer both configurations, but I personally think this is an unhelpful level of granularity to offer to users, since we should work to make the concept of Severity a simple and monolithic one.

My current view on this is:

• Configured severity refers to rules to run
• Rules should be marked with the highest severity they can emit
• We should run all such rules and then filter out any diagnostics of unwanted severities

But that's just while I'm thinking on it now, and I'd like to open this question up for discussion

Start by reading the linked PSScriptAnalyzer issue #1610 and the severity discussion here. Determine whether the project has an existing rule or diagnostic severity configuration to anchor the decision; done would require an agreed severity model and an implementation scope, neither of which this issue specifies.