Adding profiling and performance analysis during benchmarking

We need to continue improving universal benchmarking and optimization capabilities in CM for different OS and hardware targets:

• For compiled code (C/C++ ...) we improve the following CM scripts

  • CM scripts:
    • compile-program
    • benchmark-program
    • benchmark-program-mlperf
  • TBD
    • support for gprof/oprofile/hardware counters
    • universal support to pin threads (numactl)
    • expose internal profiling info from ML frameworks and run-times if/when available (onnx, TFLite ...)
  • Sample apps:
    • Corner detection (C)
    • MLPerf ONNX C++ harness
    • MLPerf TFLite C++ harness

• For Python:

  • create CM script with a python package to collect various profiling info (memory utilization, etc) particularly to analyze ML/AI models (was asked by ABTF).
  • Collect function-level profiling

• Add support for universal performance analysis to CM experiment:

  • Aggregate profiling from multiple runs and perform stat analysis (variation, min/max, phases, etc)
  • Visualize experiments

Start by reading the compile-program, benchmark-program, benchmark-program-mlperf, and automation/experiment scripts, then compare the listed C/C++ sample apps with the Python profiling requirements. The issue is complete only when profiling collection, multi-run analysis, and experiment visualization are defined and implemented across the requested OS and hardware targets.