setting n_gpu_layers to 0 or -1 still tries to use the gpu llamaindex

torch.cuda.OutOfMemoryError: HIP out of memory. Tried to allocate 224.00 MiB. GPU 0 has a total capacty of 23.98 GiB of which 44.00 MiB is free. Of the allocated memory 23.68 GiB is allocated by PyTorch, and 1.14 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_HIP_ALLOC_CONF

Look at the GPU layer configuration in the llama-cpp-python bindings, likely in the model loading or context initialization code. The issue suggests that setting n_gpu_layers to 0 or -1 does not disable GPU usage as expected. Start by tracing where n_gpu_layers is passed to the underlying llama.cpp library and check the HIP/PyTorch memory allocation paths. Verify the fix by running a minimal script that reproduces the out-of-memory error.

Written by the indexing model from the issue text.