[Question] How to run MLPerf with dual GPU

Hi everyone,

I’m currently experimenting with MLPerf Client on a system with the following hardware:

• CPU: Intel Core Ultra 9 285K
• GPU: 2× Intel Arc Pro B60
• NPU: Intel AI Boost

I’m running the LLM scenarios (Llama 3.1 and Phi 3.5) using the Native OpenVINO backend.

By default, the provided configs use "device_type": "GPU", which appears to target a single GPU. I’ve also tested "MULTI:GPU", but it’s not clear how effectively both GPUs are being utilized for these workloads, especially with LLM inference.

My goal is to understand the best way to leverage both GPUs for benchmarking. Specifically:

• Are there any config parameters or best practices to ensure both GPUs are actively used?
• Would running multiple parallel instances (e.g., with different device IDs) provide more reliable scaling?
• Are there known limitations when using multi-GPU with LLMs in this context?

Any guidance or examples would be greatly appreciated.
Thanks in advance for your support!

Start with the provided LLM scenario configs and compare the documented behavior of device_type: "GPU" and "MULTI:GPU" under the Native OpenVINO backend. Review the questions about device IDs, parallel instances, scaling, and LLM limitations, then document verified configuration guidance or examples for using both GPUs.