Make the model run on lower GPU at the expense of time/speed

### 🚀 The feature, motivation and pitch

Add command-line flags (e.g., --max-model-len) that allow users to:
Manually cap the context window (e.g., to 4096 or 8192) to fit within smaller VRAM buffers.

Trade off processing speed or context size to allow the model to run on 12GB, 10GB, or 8GB GPUs.

This would make the tool accessible to a much wider range of researchers and hobbyists who are running local workstations rather than server-grade hardware.

### Alternatives

_No response_

### Additional context

_No response_