IPEX unable to work keep offload to CPU

OS: Windows 11 Pro
WSL: Ubuntu-22.04
GPU: A770 16GB
Docker Container: intel/intel-extension-for-pytorch:xpu-flex-2.0.110-xpu (https://hub.docker.com/r/intel/intel-extension-for-pytorch)
Docker commend: docker run -it --name rvc -v D:\Download:/home/user/data --device /dev/dxg -v /usr/lib/wsl:/usr/lib/wsl -p 7897:7897 intel/intel-extension-for-pytorch:xpu-flex-2.0.110-xpu

Running clinfo | grep "Device Name" confirms that Docker detects my GPU:

user@81e518f4eb2e:/$ clinfo | grep "Device Name"
  Device Name                                     Intel(R) Graphics [0x56a0]
  Device Name                                     Intel(R) Graphics [0x3e92]
  Device Name                                     Intel(R) FPGA Emulation Device
  Device Name                                     Intel(R) Core(TM) i5-8400 CPU @ 2.80GHz
    Device Name                                   Intel(R) Graphics [0x56a0]
    Device Name                                   Intel(R) Graphics [0x56a0]
    Device Name                                   Intel(R) Graphics [0x56a0]

I also ran the following command to check if IPEX is working properly:
python -c "import torch; import intel_extension_for_pytorch as ipex; print(torch.__version__); print(ipex.__version__); [print(f'[{i}]: {torch.xpu.get_device_properties(i)}') for i in range(torch.xpu.device_count())];"

/home/user/env/lib/python3.10/site-packages/torchvision/io/image.py:13: UserWarning: Failed to load image Python extension: ''If you don't plan on using image functionality from `torchvision.io`, you can ignore this warning. Otherwise, there might be something wrong with your environment. Did you have `libjpeg` or `libpng` installed before building `torchvision` from source?
  warn(
2.0.1a0+cxx11.abi
2.0.110+xpu
[0]: _DeviceProperties(name='Intel(R) Graphics [0x56a0]', platform_name='Intel(R) Level-Zero', dev_type='gpu, support_fp64=0, total_memory=12992MB, max_compute_units=512, gpu_eu_count=512)
[1]: _DeviceProperties(name='Intel(R) Graphics [0x3e92]', platform_name='Intel(R) Level-Zero', dev_type='gpu, support_fp64=1, total_memory=13043MB, max_compute_units=24, gpu_eu_count=24)

Issue: IPEX Keeps Offloading to CPU
Despite following the official setup instructions for IPEX, it continues to offload workloads to the CPU instead of utilizing the GPU. Any insights or suggestions on resolving this?

the terminal do show my gppu is found

During step 2 feature extraction

2025-02-27 14:21:15 | INFO | __main__ | Execute: "/home/user/env/bin/python" infer/modules/train/extract_feature_print.py xpu:0 2 0 0 "/home/user/Retrieval-based-Voice-Conversion-WebUI/logs/mi-test" v2 True
2025-02-27 14:21:15 | INFO | __main__ | Execute: "/home/user/env/bin/python" infer/modules/train/extract_feature_print.py xpu:0 2 1 1 "/home/user/Retrieval-based-Voice-Conversion-WebUI/logs/mi-test" v2 True
infer/modules/train/extract_feature_print.py xpu:0 2 1 1 /home/user/Retrieval-based-Voice-Conversion-WebUI/logs/mi-test v2 True
infer/modules/train/extract_feature_print.py xpu:0 2 0 0 /home/user/Retrieval-based-Voice-Conversion-WebUI/logs/mi-test v2 True
exp_dir: /home/user/Retrieval-based-Voice-Conversion-WebUI/logs/mi-test
exp_dir: /home/user/Retrieval-based-Voice-Conversion-WebUI/logs/mi-test
load model(s) from assets/hubert/hubert_base.pt
load model(s) from assets/hubert/hubert_base.pt
2025-02-27 14:21:17 | INFO | fairseq.tasks.hubert_pretraining | current directory is /home/user/Retrieval-based-Voice-Conversion-WebUI
2025-02-27 14:21:17 | INFO | fairseq.tasks.hubert_pretraining | current directory is /home/user/Retrieval-based-Voice-Conversion-WebUI
2025-02-27 14:21:17 | INFO | fairseq.tasks.hubert_pretraining | HubertPretrainingTask Config {'_name': 'hubert_pretraining', 'data': 'metadata', 'fine_tuning': False, 'labels': ['km'], 'label_dir': 'label', 'label_rate': 50.0, 'sample_rate': 16000, 'normalize': False, 'enable_padding': False, 'max_keep_size': None, 'max_sample_size': 250000, 'min_sample_size': 32000, 'single_target': False, 'random_crop': True, 'pad_audio': False}
2025-02-27 14:21:17 | INFO | fairseq.tasks.hubert_pretraining | HubertPretrainingTask Config {'_name': 'hubert_pretraining', 'data': 'metadata', 'fine_tuning': False, 'labels': ['km'], 'label_dir': 'label', 'label_rate': 50.0, 'sample_rate': 16000, 'normalize': False, 'enable_padding': False, 'max_keep_size': None, 'max_sample_size': 250000, 'min_sample_size': 32000, 'single_target': False, 'random_crop': True, 'pad_audio': False}
2025-02-27 14:21:17 | INFO | fairseq.models.hubert.hubert | HubertModel Config: {'_name': 'hubert', 'label_rate': 50.0, 'extractor_mode': default, 'encoder_layers': 12, 'encoder_embed_dim': 768, 'encoder_ffn_embed_dim': 3072, 'encoder_attention_heads': 12, 'activation_fn': gelu, 'layer_type': transformer, 'dropout': 0.1, 'attention_dropout': 0.1, 'activation_dropout': 0.0, 'encoder_layerdrop': 0.05, 'dropout_input': 0.1, 'dropout_features': 0.1, 'final_dim': 256, 'untie_final_proj': True, 'layer_norm_first': False, 'conv_feature_layers': '[(512,10,5)] + [(512,3,2)] * 4 + [(512,2,2)] * 2', 'conv_bias': False, 'logit_temp': 0.1, 'target_glu': False, 'feature_grad_mult': 0.1, 'mask_length': 10, 'mask_prob': 0.8, 'mask_selection': static, 'mask_other': 0.0, 'no_mask_overlap': False, 'mask_min_space': 1, 'mask_channel_length': 10, 'mask_channel_prob': 0.0, 'mask_channel_selection': static, 'mask_channel_other': 0.0, 'no_mask_channel_overlap': False, 'mask_channel_min_space': 1, 'conv_pos': 128, 'conv_pos_groups': 16, 'latent_temp': [2.0, 0.5, 0.999995], 'skip_masked': False, 'skip_nomask': False, 'checkpoint_activations': False, 'required_seq_len_multiple': 2, 'depthwise_conv_kernel_size': 31, 'attn_type': '', 'pos_enc_type': 'abs', 'fp16': False}
2025-02-27 14:21:17 | INFO | fairseq.models.hubert.hubert | HubertModel Config: {'_name': 'hubert', 'label_rate': 50.0, 'extractor_mode': default, 'encoder_layers': 12, 'encoder_embed_dim': 768, 'encoder_ffn_embed_dim': 3072, 'encoder_attention_heads': 12, 'activation_fn': gelu, 'layer_type': transformer, 'dropout': 0.1, 'attention_dropout': 0.1, 'activation_dropout': 0.0, 'encoder_layerdrop': 0.05, 'dropout_input': 0.1, 'dropout_features': 0.1, 'final_dim': 256, 'untie_final_proj': True, 'layer_norm_first': False, 'conv_feature_layers': '[(512,10,5)] + [(512,3,2)] * 4 + [(512,2,2)] * 2', 'conv_bias': False, 'logit_temp': 0.1, 'target_glu': False, 'feature_grad_mult': 0.1, 'mask_length': 10, 'mask_prob': 0.8, 'mask_selection': static, 'mask_other': 0.0, 'no_mask_overlap': False, 'mask_min_space': 1, 'mask_channel_length': 10, 'mask_channel_prob': 0.0, 'mask_channel_selection': static, 'mask_channel_other': 0.0, 'no_mask_channel_overlap': False, 'mask_channel_min_space': 1, 'conv_pos': 128, 'conv_pos_groups': 16, 'latent_temp': [2.0, 0.5, 0.999995], 'skip_masked': False, 'skip_nomask': False, 'checkpoint_activations': False, 'required_seq_len_multiple': 2, 'depthwise_conv_kernel_size': 31, 'attn_type': '', 'pos_enc_type': 'abs', 'fp16': False}
move model to cpu
all-feature-64
move model to cpu
all-feature-63
now-64,all-0,100_1.wav,(103, 768)
now-63,all-0,10_1.wav,(129, 768)
now-63,all-6,21_1.wav,(137, 768)
now-64,all-6,20_1.wav,(198, 768)
now-64,all-12,2_2.wav,(117, 768)
now-63,all-12,30_1.wav,(124, 768)

Even for training i notice it always use CPU instead of my dGPU

Start with infer/modules/train/extract_feature_print.py and the feature-extraction command shown in the logs. Trace where the Hubert model is placed and why the run reports “move model to cpu,” then compare that behavior with the xpu:0 device argument. Done means feature extraction and training use the detected GPU rather than falling back to the CPU.