I suggest adding an "Accuracy" property to DiagnosticRecord.

Suggestion

Add an "Accuracy" property to DiagnosticRecord that will state the probability of the diagnostic being correct.

Benefits for the Users

Users can filter and sort diagnostics based on their accuracy.

    invoke-scriptanalyzer |
        where {$_.accuracy -gt 0.75} |
        sort rulename, accuracy -descending

Benefits for the Rule Writers

Rule writers can introduce new rules sooner by introducing rules with a low accuracy, and then later increase the accuracy as the rule is refined.

For example, the "UseShouldProcessForStateChangingFunctions" rule right now is based only on the function name, so the rule can be assigned with an accuracy of, let's say, 0.25. If the rule is improved with better heuristics, then its accuracy can be increased. If the default of Invoke-ScriptAnalyzer doesn't show diagnostics with an accuracy of less than 0.50, then users won't have to suppress the "UseShouldProcessForStateChangingFunctions" while the rule is still in its initial stages.

This allows rule writers to get feedback on their rules while the rules are still being refined without adding noise to Invoke-ScriptAnalyzer results.

Benefits for the Rules

The rules can have multiple diagnostic accuracies.

For example, the "PossibleIncorrectComparisonWithNull" will be more useful if it can output diagnostics with different accuracies based on context.

A line such as:

    if ($a -ne $null)

should output a diagnostic for this rule with a high accuracy because "$a -ne $null" is very likely to be a compare operation instead of a filter operation.

However, a line such as:

    $a = $b -ne $null

should output a diagnostic for this rule with a low accuracy because "$b -ne $null" can either be a compare operation or a filter operation, and it will be very difficult to assert that the user violated the rule.

Now, the rule doesn't have to be hardcoded with the compromise of when to generate diagnostics because it can generate all diagnostics with different accuracy levels.

Comece rastreando DiagnosticRecord e o caminho de saída de Invoke-ScriptAnalyzer; a issue não nomeia arquivos de origem nem testes. Determine como a precisão deve ser representada, produzida pelas regras e exposta para filtragem e ordenação. Considera-se concluído quando a propriedade e sua semântica forem compatíveis de forma consistente com os cenários de diagnóstico propostos.