Memory Management Issues

Nobody has claimed this yet.

**Severity:** High - Will bite you in production

The code itself admits there are memory issues. From `litgpt/utils.py:316`:

```python
# as a workaround hack, the cross entropy computation is chunked to force it to
# deallocate on the go, reducing the memory spike's magnitude
```

When the developers are calling their own code a "workaround hack", that's not great.

### Specific problems:

**KV Cache management is sketchy:**
```python
# litgpt/model.py:66
if self.mask_cache is not None and self.mask_cache.shape[-1] < value:
print(
f"Warning: KV cache has length {self.mask_cache.shape[-1]} < {value}..."
)
```

This just prints a warning at runtime. It should either:
- Fix the cache size automatically
- Raise an exception BEFORE trying to use it
- Not get into this state in the first place

**The chunked cross-entropy thing:**

Yes, it works. But it's papering over a real issue - the backward pass is allocating way more memory than it should. This suggests either:
- Something's not getting deallocated properly
- The computation graph is keeping references it shouldn't
- The CUDA memory allocator isn't being triggered when it should

### What happens in practice:

- Random OOMs during training that are hard to reproduce
- Memory usage creeps up over time in long-running inference servers
- If you try to use the full context length with large batch sizes, good luck

### Fix it properly:

1. Profile the memory with PyTorch's memory profiler - find the actual leak
2. Make KV cache lifecycle explicit - init, use, clear, destroy
3. Add a proper memory budget system instead of chunking hacks
4. Test with full context lengths and realistic batch sizes

Start by profiling the memory behavior around the chunked cross-entropy workaround in litgpt/utils.py:316 and the KV-cache warning in litgpt/model.py:66. Reproduce the reported training and inference memory growth with full context lengths and realistic batch sizes, then use PyTorch's memory profiler to identify the cause. Done means the leak or allocation problem is addressed, KV-cache lifecycle is explicit, and the reported OOM scenarios are tested.

Written by the indexing model from the issue text.