DJL Serving / vLLM — LLaVA tokenizer.json incompatibility causes endpoint startup failure

**PySDK Version**
- [ ] PySDK V3 (3.4.0)

**Describe the bug**
DJL Serving / vLLM — LLaVA tokenizer.json incompatibility causes endpoint startup failure

When deploying a LLaVA multimodal model to a SageMaker endpoint using the djl-inference:0.29.0-lmi11.0.0-cu124 container with OPTION_ROLLING_BATCH=vllm, the Python engine process crashes immediately on startup with the following error:
Exception: data did not match any variant of untagged enum ModelWrapper at line 277156 column 3
The crash occurs inside the Rust tokenizers library when attempting to deserialize tokenizer.json. Two incompatibilities are present between tokenizer files produced by recent versions of the HuggingFace tokenizers library and the version bundled inside the DJL 0.29.0 container:
• The tokenizer.json BPE model block contains an ignore_merges field introduced in tokenizers >= 0.14.0 that the container's older Rust deserializer does not recognise, causing the entire ModelWrapper enum deserialization to fail.
• The merges array uses the newer list-of-lists format (e.g. ["▁", "t"]) instead of the legacy space-joined string format expected by older tokenizers (e.g. "▁ t").
Either incompatibility alone is sufficient to trigger the crash. The endpoint never becomes healthy and SageMaker eventually times out the deployment.

**To reproduce**
1. Package a LLaVA model (e.g. llava-hf/llava-1.5-7b-hf) whose tokenizer.json was saved with tokenizers >= 0.14.0 into a model.tar.gz and upload to S3.
2. Run the deployment script below:
import boto3
from sagemaker.core.helper.session_helper import Session
from sagemaker.serve.model_builder import ModelBuilder
from sagemaker.serve.mode.function_pointers import Mode
from datetime import datetime

region = "us-east-1"
sm_s3_model_path = "s3:///path/model.tar.gz"
sm_role = "arn:aws:iam:::role/"

boto_session = boto3.Session(region_name=region)
sagemaker_session = Session(boto_session=boto_session)
image_uri = (
"763104351884.dkr.ecr.us-east-1.amazonaws.com/"
"djl-inference:0.29.0-lmi11.0.0-cu124"
)

builder = ModelBuilder(
image_uri=image_uri,
s3_model_data_url=sm_s3_model_path,
role_arn=sm_role,
sagemaker_session=sagemaker_session,
env_vars={
"HF_MODEL_ID": "/opt/ml/model",
"OPTION_ROLLING_BATCH": "vllm",
"TENSOR_PARALLEL_DEGREE": "1",
"OPTION_DTYPE": "fp16",
"OPTION_MAX_MODEL_LEN": "4096",
"OPTION_TRUST_REMOTE_CODE": "true",
"OPTION_TASK": "text-generation",
},
instance_type="ml.g5.2xlarge",
mode=Mode.SAGEMAKER_ENDPOINT,
)
builder.build(role_arn=sm_role, sagemaker_session=sagemaker_session)
predictor = builder.deploy(
endpoint_name=f"llava-{int(datetime.now().timestamp())}",
initial_instance_count=1,
instance_type="ml.g5.2xlarge",
container_timeout_in_seconds=600,
)
3. Observe that the container exits with the ModelWrapper deserialization exception before the endpoint becomes InService.

**Expected behavior**
The DJL container should either:
• Accept tokenizer.json files produced by modern versions of the HuggingFace tokenizers library (including the ignore_merges field and list-format merges), OR
• Emit a clear, actionable error message indicating that the tokenizer.json format is incompatible and specifying the maximum supported tokenizers library version.
The endpoint should reach InService status and be able to serve LLaVA inference requests.

Screenshots or Logs
Key error extracted from CloudWatch / container stdout (repeated across all three retry attempts):
INFO llm_engine.py:176] Initializing an LLM engine (v0.5.3.post1) with config:
model='/opt/ml/model', dtype=torch.float16, max_seq_len=4096 ...

File ".../tokenization_utils_fast.py", line 115, in __init__
fast_tokenizer = TokenizerFast.from_file(fast_tokenizer_file)
Exception: data did not match any variant of untagged enum ModelWrapper
at line 277156 column 3

Python engine process died
[ERROR] ModelServer - Failed register workflow
Caused by: ai.djl.engine.EngineException: Failed to initialize model: prediction failure
[ERROR] ModelServer - Unexpected error
ai.djl.serving.http.ServerStartupException:
Failed to initialize startup models and workflows
The same sequence is logged on each of the three startup retries before SageMaker marks the deployment as failed.

**System information**

SageMaker Python SDK version | PySDK V3 (3.4.0) — sagemaker.serve.model_builder.ModelBuilder
-- | --
Framework / Algorithm | DJL Serving 0.29.0 with vLLM 0.5.3.post1 rolling batch
Framework version | djl-inference:0.29.0-lmi11.0.0-cu124
Python version | 3.9 (container default)
CPU or GPU | GPU — ml.g5.2xlarge (1× NVIDIA A10G, 24 GB VRAM)
Custom Docker image | N — official AWS DJL LMI image

**Additional context**
• The DJL log correctly identifies modelType: llava but the OPTION_TASK=text-generation override may also conflict with LLaVA's multimodal requirements. Removing this env var is recommended.
• A secondary (non-fatal) CUDA compatibility warning is also logged: the container's CUDA 12.4 compat package expects a driver <= 550.127.08 but the host driver is 535.288.01. This does not cause the crash but may affect future GPU operations.
• The container retries startup three times before giving up, logging the identical tokenizer error each time, resulting in a confusingly long CloudWatch log.

从提供的 SageMaker ModelBuilder deployment script 开始，并使用 djl-inference:0.29.0-lmi11.0.0-cu124 镜像重现该故障。通过 llm_engine.py 和 tokenization_utils_fast.py 跟踪启动日志，然后确定此仓库是控制失败 tokenizer 的兼容性，还是仅报告该问题。完成的标准是明确责任归属和受支持的修复方案；如果问题属于此仓库，则应提供回归测试或清晰、可执行的错误。

由索引模型根据 Issue 内容生成。