The requirements for more VRAM seem much higher than advertised.

The model cant run on my PC with this error:
torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 30.28 GiB. GPU 0 has a total capacity of 8.00 GiB of which 3.79 GiB is free. Of the allocated memory 2.83 GiB is allocated by PyTorch, and 302.09 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation.

Why is it trying to allocate 30.28 GB of material?
I have tried enabling expandable segments, but its still not working.
Is there a way to reduce batch size in the model?

Start by reproducing the reported PyTorch CUDA out-of-memory error with the model on an 8 GB GPU, including the attempted PYTORCH_CUDA_ALLOC_CONF setting. Inspect how the model determines memory use and whether batch size is configurable; done means the cause of the 30.28 GiB allocation and any supported lower-memory configuration are documented or corrected.

Written by the indexing model from the issue text.