OutOfMemoryError - but still some free space ?

I'm using SDXL + openposeXL + IPAdapter and some loras which of course are taking a lot of VRAM.
Often my workflows are trying to allocate more VRAM than my RTX3090 24GB actually has.

But I'm curious, because sometimes messages like these appear:
```
torch.cuda.OutOfMemoryError: Allocation on device 0 would exceed allowed memory. (out of memory)
Currently allocated : 21.55 GiB
Requested : 952.00 MiB
Device limit : 24.00 GiB
Free (according to CUDA): 0 bytes
PyTorch limit (set by user-supplied memory fraction)
: 17179869184.00 GiB

Prompt executed in 188.48 seconds
```
What caught my attention is that the "Currently allocated" + "Requested" is still way less than the "Device limit" of 24GB.
(I'm already running in lowvram mode)

Any ideas?

Thanks in advance and kind regards!