NameError: name 'torch' is not defined

using the latest pull, I have getting the error below when I try to run it with (CUDA):
python3 -m fastchat.serve.cli --model-path lmsys/fastchat-t5-3b-v1.0 --load-8bit --cpu-offloading
It is working correctly without --cpu-offloading. Is it expected? My understanding from the readme is that we could mix the 2 (load to the GPU and the CPU)

The Stack trace is:
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/home/shuther/devProjects/FastChat/fastchat/serve/cli.py", line 304, in <module>
    main(args)
  File "/home/shuther/devProjects/FastChat/fastchat/serve/cli.py", line 227, in main
    chat_loop(
  File "/home/shuther/devProjects/FastChat/fastchat/serve/inference.py", line 361, in chat_loop
    model, tokenizer = load_model(
                       ^^^^^^^^^^^
  File "/home/shuther/devProjects/FastChat/fastchat/model/model_adapter.py", line 320, in load_model
    model, tokenizer = adapter.load_model(model_path, kwargs)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/shuther/devProjects/FastChat/fastchat/model/model_adapter.py", line 729, in load_model
    model = AutoModelForSeq2SeqLM.from_pretrained(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/shuther/mambaforge/lib/python3.11/site-packages/transformers/models/auto/auto_factory.py", line 566, in from_pretrained
    return model_class.from_pretrained(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/shuther/mambaforge/lib/python3.11/site-packages/transformers/modeling_utils.py", line 3480, in from_pretrained
    ) = cls._load_pretrained_model(
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/shuther/mambaforge/lib/python3.11/site-packages/transformers/modeling_utils.py", line 3870, in _load_pretrained_model
    new_error_msgs, offload_index, state_dict_index = _load_state_dict_into_meta_model(
                                                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/shuther/mambaforge/lib/python3.11/site-packages/transformers/modeling_utils.py", line 751, in _load_state_dict_into_meta_model
    set_module_quantized_tensor_to_device(
  File "/home/shuther/mambaforge/lib/python3.11/site-packages/transformers/integrations/bitsandbytes.py", line 58, in set_module_quantized_tensor_to_device
    if old_value.device == torch.device("meta") and device not in ["meta", torch.device("meta")] and value is None:
                           ^^^^^
NameError: name 'torch' is not defined

Reproduce the reported command with --load-8bit and --cpu-offloading, then trace the call from fastchat/serve/cli.py through fastchat/serve/inference.py and fastchat/model/model_adapter.py to the failing Transformers integration. Done means the documented CUDA configuration no longer raises the reported NameError, or its compatibility requirements are clearly established.