v4.0 Inference auditing procedure postmortem

Specifically for the nominated auditing selection procedure. To prevent misleading information during the nominated auditing selection procedure, it's essential to prioritize transparency and accuracy in the data provided. An issue arose where a significant performance difference of over 20% for L40S was mentioned without specifying the comparison context. This lack of specificity resulted in a mistake during the auditing process, where the comparison inadvertently paired top results with outlier low results. Consequently, this led to voters making decisions based on incorrect information.

To address this, we propose using a table format similar to the one below to provide relevant data along with the nomination:

This will help avoid mistakes, which can result in misleading voters. It will also contribute to making the correct auditing selection, ensuring that resources are allocated appropriately.

Thanks,
Frank

The issue describes a nominated auditing selection procedure and proposes presenting nomination data in a comparison table, but it identifies no repository file, test, or entry point. Start by locating where this procedure and nomination information are maintained; done means the comparison context is explicit and voters are not led to compare top results with outliers.