Running local LLMs

## Brief overview of Ollama, vLLM and SGLang

To use **open-weight models** on your machine, you have three main options: `Ollama`, `vLLM`, and `SGLang`.
Each engine handles requests differently. The diagram below shows the differences and the main techniques behind each engine.

* Ollama: Ollama is best for local dev, prototyping, and laptop-scale hardware. The architecture is inherently **sequential**. A local user calls the OpenAI-compatible API, and requests line up in a FIFO queue. Then Ollama runs a pre-quantized GGUF model, a compressed format it pulls, and the response comes back to the user.

* vLLM: vLLM is best for high-traffic serving, max GPU utilization, and thousands of concurrent requests. Many users hit the server at once, and continuous batching slots new requests into the running batch instead of making them wait for it to finish. **PagedAttention** stores the KV cache, the memory a model keeps for tokens it has already processed. The PagedAttention maps the OS memory pages to vLLM memory blocks.

* SGLang: SGLang is best for AI agents and tool loops, multi-turn chats, and JSON/regex outputs. The most commun example is when the workflow involves using **repeated context**, like a static system prompt or a large RAG documents, during a CI workflow. Agents and multi-turn chats send requests whose prompts overlap heavily. A prefix-aware scheduler routes them through the RadixAttention cache, a **radix tree** that reuses every shared prefix instead of recomputing it.

> [!NOTE]
> This is for the big picture. Needs to be continued with a bit more in deep insights...

Start with the existing issue text and its linked architecture diagram, then review how Ollama, vLLM, and SGLang are described. Extend the overview with the deeper insights requested in the note, keeping the comparison understandable for someone choosing a local LLM engine. Done means the documented differences and techniques are expanded beyond the current big-picture summary.

Written by the indexing model from the issue text.