Cache Statistics in OpenVINO Accuracy Restorer

Description

In the OpenVINO backend, statistics collection inside quantize_with_accuracy_control_impl can be redundant when hyperparameter tuning is enabled.

Currently, the same statistics may be recalculated during both the initial quantization pass and the subsequent tuning/restoration steps, which can double the runtime of an already expensive process.

Location

quantize_model.py:L254

Problem Summary

• Statistics are recalculated unnecessarily during hyperparameter tuning
• Leads to slower quantization
• Wastes compute and reduces user experience for large models
• The operation is expensive and redundant

Suggested Fix

• Implement caching of statistics collected during the initial quantization pass
• Reuse cached statistics for subsequent tuning or restoration steps
• This avoids redundant computation and improves runtime

Example idea:

# Pseudocode
if statistics_cache_exists:
    use_cached_statistics()
else:
    collect_statistics()
    save_to_cache()

Expected Benefits

• Faster hyperparameter tuning and accuracy restoration
• Reduced computation overhead
• Improved user experience and model quantization workflow
• Maintains correctness of accuracy-aware quantization