Qwen 3.6 Tensor Parallelism with FP8 KV cache instructions are missing along with instructions to maximize concurrent prompts

## Programs

We need highest tokens/sec throughput or best accuracy for long-context, but you need a Masters Degree in data science to understand this. I have a Gigabyte Z890 Aero G mobo with PCI x8/x8 5.0 bifurcation, Core 7 Ultra 245K, and 64GB DDR5 6400, and the #1 model we want to run on this GPU is Qwen 3.6 27B, and I might buy a third just to run 35B A3B. I have my models downloaded to my NAS, and I need to pass the model into the Container. I had to use Hermes Agent and ChatGPT to configure the server. Just go to the page and do a CTRL+F and search for "Qwen" or "3.6" or "Qwen 3.6", which has no hits. Making me search for "Qwen3.6" is not acceptable. I paid $2000, I'm entitled to a section in the instructions for my use case, we all want tensor parallelism working now, then the problem becomes evident: your instructions are for Qwen2.5-7B-Instruct, no one wants to run that, literally no one. When I go to Intel's website, there is no information about LLM Scalar. I'm only here because I'm an engineer, but most of your customers are really disappointed their cards they paid $1000 require a bachelors degree in computer science to set up or Masers degree to run optimally. I paid $2000 for two Arc B70 pro for the sole reason that on YouTube I was told the software stack was mature enough that it's not a paperweight. I was shocked to find that the instructions were non existent and driver download page had no lines to LLM Scalar. This is the most unprofessional hardware product I've bought. Almost every product I've bought at least has a web page on the company site or at least an article that walks you through it. Additionally, there is no information about MoE vs dense models using tensor parallelism, they have no clue MoE models don't work well for TP. due to unpredictable PCIe timing. Also, I've had a lot of glitches with the vLLM version recommended for Qwen 3.6, it's outputted and lot of gibberish and all !. Right now the server is down, I'm out of my $20/mo ChatGPT credits, but I shouldn't need that, the docs must contain correct information.

## Solution

Your instructions to meet customer needs must contain the instructions to run two Arc B70 tensor parallelism using the most powerful model (Qwen 3.6 27B right now) using FP8 weights and Unsloth Q4_K_XL using as much VRAM as possible and running as many concurrent prompts as possible. Your instructions require the information that $20/mo ChatGPT and Hermes Agent will fix their configuration problems if you point the LLM to the official docs. Add "Qwen 3.6" with the space to the docs to make it easy for people to get to the right spot. The newest and most popular models in LM Studio require the docs to be updated with instructions for marketing SEO search term reasons. When I search for "Qwen 3.6 Arc B70 token rate", or any trending model, this repo should be in the search results. By adding all the current models, and a check or x if it's working or not to this repo, you're maximizing your web traffic. People just use this card with LM Studio, they have no clue what vLLM is, chances are FP8 doesn't ring a bell, so the docs must contain an explanation of FP8 vs FP16 KV cache context. You need shell script that contains not only the server reset instructions, but all too a link to the docs. Here is the script ChatGPT created and I edited, but it's not working with FP8 KV cache context.

```bash
# Official docs at https://github.com/intel/llm-scaler/blob/main/vllm/README.md
# Intel LLM Scaler Qwen3.6 settings for concurrent interactive work.
# Online FP8 quantizes weights. FP8 KV uses dynamically calculated K/V scales
# with the XPU-safe Q-scale behavior patched above. Preserve the model's full
# 262K FP KV cache context window; paged KV allocation still permits shorter concurrent
# requests to share the cache pool.
docker exec -d "$CONTAINER" bash -lc "
export VLLM_ALLOW_LONG_MAX_MODEL_LEN=1
export VLLM_WORKER_MULTIPROC_METHOD=spawn
export VLLM_OFFLOAD_WEIGHTS_BEFORE_QUANT=1
export ZE_AFFINITY_MASK=0,1
export VLLM_ATTENTION_BACKEND=TRITON_ATTN
exec vllm serve '$MODEL_PATH' \\
--port '$PORT' \\
--host 0.0.0.0 \\
--gpu-memory-utilization 0.95 \\
--max-num-batched-tokens 8192 \\
--max-num-seqs 4 \\
--max-model-len 262144 \\
--block-size 64 \\
--dtype float16 \\
--served-model-name '$MODEL_NAME' \\
--tensor-parallel-size 2 \\
--quantization fp8 \\
--kv-cache-dtype fp8 \\
--calculate-kv-scales \\
--swap-space 8 \\
--enforce-eager \\
--trust-remote-code \\
--disable-log-requests \\
--enable-auto-tool-choice \\
--tool-call-parser qwen3_coder \\
--reasoning-parser qwen3 \\
--default-chat-template-kwargs '{\"enable_thinking\":false}' \\
--enable-prefix-caching \\
> '$LOG_FILE' 2>&1
"

echo "vLLM restart initiated; waiting for API readiness. Log: $LOG_FILE"
# Allow the detached shell to exec the final Python process.
sleep 3
for _ in $(seq 1 180); do
if curl -fsS --max-time 2 "http://127.0.0.1:${PORT}/v1/models" >/dev/null 2>&1; then
echo "vLLM is ready on port $PORT."
exit 0
fi

if ! docker exec "$CONTAINER" bash -lc \
"pgrep -f '^/usr/bin/python3 /usr/local/bin/vllm serve' >/dev/null"; then
echo "vLLM exited during startup. Recent log output:" >&2
docker exec "$CONTAINER" bash -lc "tail -n 80 '$LOG_FILE'" >&2 || true
exit 1
fi
sleep 1
done

echo "vLLM did not become ready within 180 seconds. Recent log output:" >&2
docker exec "$CONTAINER" bash -lc "tail -n 80 '$LOG_FILE'" >&2 || true
exit 1
```

Fix this script and add it to the docs. Really I shouldn't be limited to Qwen 3.6 27B with this script.

For 35B A3B Unsloth Q4_K_XL, I get two 200K context windows using FP16 K cache and q8_8 V cache in LM Studio, so I should be able to get three 262K FP8 KV cache context windows in.

The docs must contain data bout the limitations of the VRAM. --gpu-memory-utilization 0.95 should work for a 32GB card. That is like 90% limit on a 16GB card.

In order to get the stats, we're required to monitor the vLLM server in the Docker Container, so also too add instructions how to enter the Container using "docker exec -it llms bash", and you should probably name the containers llms (LLM Scalar).