SDXL CLIPTextEncode: User specified autocast device_type must be 'cuda' or 'cpu'

`python main.py --normalvram --use-quad-cross-attention --auto-launch --disable-smart-memory --preview-method latent2rgb --dont-upcast-attention` (model is automatically loaded in lowvram mode due to 4GB of VRAM.)

```
Total VRAM 4096 MB, total RAM 15943 MB
Set vram state to: NORMAL_VRAM
Disabling smart memory management
Device: cuda:0 AMD Radeon RX 570 Series : native
VAE dtype: torch.float32
disabling upcasting of attention
```

Happened somewhat randomly when using an SDXL model where I was running my workflow without any issues, then this started happening more or less consistently.

Happens with other TextEncode nodes from custom nodes as well.

```
Error occurred when executing CLIPTextEncode:

User specified autocast device_type must be 'cuda' or 'cpu'

File "/home/rabid/Desktop/comfytwoai/execution.py", line 153, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/rabid/Desktop/comfytwoai/execution.py", line 83, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/rabid/Desktop/comfytwoai/execution.py", line 76, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/rabid/Desktop/comfytwoai/nodes.py", line 56, in encode
cond, pooled = clip.encode_from_tokens(tokens, return_pooled=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/rabid/Desktop/comfytwoai/comfy/sd.py", line 120, in encode_from_tokens
cond, pooled = self.cond_stage_model.encode_token_weights(tokens)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/rabid/Desktop/comfytwoai/comfy/sdxl_clip.py", line 56, in encode_token_weights
g_out, g_pooled = self.clip_g.encode_token_weights(token_weight_pairs_g)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/rabid/Desktop/comfytwoai/comfy/sd1_clip.py", line 18, in encode_token_weights
out, pooled = self.encode(to_encode)
^^^^^^^^^^^^^^^^^^^^^^
File "/home/rabid/Desktop/comfytwoai/comfy/sd1_clip.py", line 179, in encode
return self(tokens)
^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/rabid/Desktop/comfytwoai/venv/lib/python3.11/site-packages/accelerate/hooks.py", line 165, in new_forward
output = old_forward(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/rabid/Desktop/comfytwoai/comfy/sd1_clip.py", line 150, in forward
with precision_scope(model_management.get_autocast_device(device), torch.float32):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/torch/amp/autocast_mode.py", line 201, in __init__
raise RuntimeError('User specified autocast device_type must be \'cuda\' or \'cpu\'')
```
```
Loading 1 new model
model_type EPS
adm 2816
Using split attention in VAE
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
Using split attention in VAE
missing {'cond_stage_model.clip_l.text_projection', 'cond_stage_model.clip_l.logit_scale'}
left over keys: dict_keys(['conditioner.embedders.0.logit_scale', 'conditioner.embedders.0.text_projection', 'cond_stage_model.clip_l.transformer.text_model.embeddings.position_ids'])
Requested to load SDXLClipModel
Loading 1 new model
loading in lowvram mode 445.0304231643677
ERROR:root:!!! Exception during processing !!!
ERROR:root:Traceback (most recent call last):
File "/home/rabid/Desktop/comfytwoai/execution.py", line 153, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/rabid/Desktop/comfytwoai/execution.py", line 83, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/rabid/Desktop/comfytwoai/execution.py", line 76, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/rabid/Desktop/comfytwoai/nodes.py", line 56, in encode
cond, pooled = clip.encode_from_tokens(tokens, return_pooled=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/rabid/Desktop/comfytwoai/comfy/sd.py", line 120, in encode_from_tokens
cond, pooled = self.cond_stage_model.encode_token_weights(tokens)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/rabid/Desktop/comfytwoai/comfy/sdxl_clip.py", line 56, in encode_token_weights
g_out, g_pooled = self.clip_g.encode_token_weights(token_weight_pairs_g)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/rabid/Desktop/comfytwoai/comfy/sd1_clip.py", line 18, in encode_token_weights
out, pooled = self.encode(to_encode)
^^^^^^^^^^^^^^^^^^^^^^
File "/home/rabid/Desktop/comfytwoai/comfy/sd1_clip.py", line 179, in encode
return self(tokens)
^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/rabid/Desktop/comfytwoai/venv/lib/python3.11/site-packages/accelerate/hooks.py", line 165, in new_forward
output = old_forward(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/rabid/Desktop/comfytwoai/comfy/sd1_clip.py", line 150, in forward
with precision_scope(model_management.get_autocast_device(device), torch.float32):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/torch/amp/autocast_mode.py", line 201, in __init__
raise RuntimeError('User specified autocast device_type must be \'cuda\' or \'cpu\'')
RuntimeError: User specified autocast device_type must be 'cuda' or 'cpu'
```

Changing to a SD 1.5 model on the same workflow in the same session where the error happened = works with no problems.

I think it might have to do with --normalvram and not manually specifying --lowvram for SDXL that is beyond my VRAM limit. If I set --lowvram myself for the purpose of using SDXL models, this autocast issue seems to stop. (not thoroughly tested yet) However if everything is manually set to lowvram on my system, that includes the steps of the workflow that don't have to be offloaded to RAM and ends up making things very sluggish and slow as it moves things around.