Can not use Intel Arc GPU

Nobody has claimed this yet.

When I run `python -m fastchat.serve.model_worker --model-path lmsys/vicuna-7b-v1.5 --device xpu --gpus X`, whatever the X is, or without `--gpus`, the memory will be always loaded to both CPU and integrated GPU, and calculation will be always on CPU, single threaded.

I can confirm that GPUs can be recognized by torch:
``` python
import torch
import intel_extension_for_pytorch
torch.xpu.current_device()
# 0
torch.xpu.get_device_name(0)
# 'Intel(R) UHD Graphics 770'
torch.xpu.get_device_name(1)
# 'Intel(R) Arc(TM) A770 Graphics'
```

Any help will be great🙏

Start with the `python -m fastchat.serve.model_worker` entry point and reproduce the command using `--device xpu` with and without `--gpus`. Compare the reported torch XPU devices with where model memory and computation are placed; done means inference uses the Intel Arc GPU rather than CPU execution.

Written by the indexing model from the issue text.