Power measurement via BMC / IPMI / in-band, Performance per W

I want to request that for single-system power measurement, BMC/IPMI-reported power, or in-band reported via ACPI or other methods, can be used. This is much more detailed and useful than nameplate power and can easily be obtained on modern servers that will be executing the MLPerf Storage runs.

e.g.

sudo ipmitool dcmi power reading

    Instantaneous power reading:                   314 Watts
    Minimum during sampling period:                314 Watts
    Maximum during sampling period:                314 Watts
    Average power reading over sample period:      314 Watts
    IPMI timestamp:                           06/17/2025 02:50:26 PM UTC    Sampling period:                          00001178 Seconds.
    Power reading state is:                   activated

Can filter range in node-exporter, prometheus, and grafana to get power for a specifc run (e.g. via ACPI)
node_hwmon_power_average_watt{instance="$node"}

I want to clarify what can be used with said power measurements, GB/s/W in U-Net3D would be an interesting metric. GB/s/W per SSD as well.

AI summary of current MLPerf power measurement.

MLPerf Power Measurement Rules Overview

MLPerf, managed by MLCommons, sets industry-standard rules for measuring the power consumption of machine learning (ML) systems during benchmarking. These rules are designed to ensure that power measurements are accurate, reproducible, and comparable across different hardware and software platforms.

Key Rules and Requirements for Power Measurement in MLPerf:

• Full System Power Measurement:
  Power must be measured for the entire System Under Test (SUT), not just individual components. This includes all compute nodes, interconnects, and, where relevant, cooling infrastructure for data centers. For edge and tiny devices, the measurement must cover the complete device, including any always-on peripherals^4.
• Measurement Phase:
  Power is measured specifically during the Performance phase of the benchmark, which is when the actual workload is being executed. This ensures that the reported power consumption reflects the energy used to perform ML inference or training, not just system idle or setup times^1^6.
• Certified Equipment:
  Only SPEC-certified power analyzers (meters) are permitted for official MLPerf power measurements. These analyzers must be able to measure AC wall power to capture the true energy usage of the system^3.
• Measurement Methodology:
  • The power analyzer is connected between the power source and the SUT, measuring current in series and voltage in parallel.
  • For larger systems, software-based timestamps are used to align power measurement precisely with the workload execution phase.
  • For tiny devices, hardware pins may be used to signal the start and end of inference runs^3.
  • The measurement setup and configuration must remain unchanged throughout the submission and review process to ensure replicability^1.
• Reporting and Submission:
  • Submitters must provide detailed logs of both performance and power measurements.
  • The platform description, power analyzer settings, and power management configurations must be disclosed.
  • The measured power is only valid for the specific benchmark run and must be reported alongside the corresponding performance metrics (e.g., queries per second, samples per second)^1.
• Reproducibility and Availability:
  • The SUT and measurement setup must be available for review upon request.
  • Submitters must be able to reproduce power runs during the review period to resolve any discrepancies^1.
• Accuracy and Frequency:
  • Power measurements should be reported at a frequency of at least one measurement per second.
  • The measurement process targets an accuracy within 5%^2.
• Scope of Measurement:
  • For training benchmarks, all nodes participating in the computation must be measured.
  • Interconnect power must be measured or estimated.
  • Power for other components (e.g., storage, networking not directly involved in the workload) is not required unless they are integral to the SUT’s operation^2.

Summary Table: MLPerf Power Measurement Rules

Additional Notes

• MLPerf Power rules are periodically updated, so always consult the latest official documentation for the most current requirements^1^3^6.
• Optional power measurement is available for some benchmarks (e.g., MLPerf Inference: Tiny), but when performed, it must follow the same strict rules for consistency and comparability^5.

By adhering to these rules, MLPerf ensures that power and energy efficiency metrics are meaningful, reproducible, and fair across the rapidly evolving landscape of ML hardware and software platforms.

No repository files, tests, or implementation entry points are named. Start by reading the linked MLPerf power-measurement policies and reviewing the example ipmitool and node-exporter metrics; done requires an agreed measurement method, reporting scope, and definition for performance-per-watt metrics.