[Bug]: Redundant KV host cache tier on DGX Spark unified-memory systems

System Info

• GPU: NVIDIA GB10 / DGX Spark class integrated GPU (SM121), 128 GB unified CPU/GPU memory
• Also applies to other integrated GPU systems where CPU and GPU share one physical memory pool
• TensorRT-LLM: current main behavior as addressed by PR #12301

Description

On unified-memory systems such as DGX Spark, CPU and GPU share the same physical LPDDR memory pool. The KV cache manager currently treats GPU memory and host memory as separate cache tiers. When kv_cache_config.host_cache_size is nonzero, TensorRT-LLM can reserve a secondary host-cache budget even though there is no separate host DRAM tier to offload to.

That makes the secondary KV tier redundant on these systems and can lead to unnecessary logical offload/onboard behavior, confusing memory accounting, and wasted startup/runtime work. cudaMemGetInfo already reports the unified pool, so host-cache bytes must not be added as a separate budget on integrated GPUs.

Expected behavior

On integrated/unified-memory GPUs:

• Detect that CPU and GPU share one physical memory pool.
• Fold host_cache_size to zero instead of allocating/accounting a separate secondary cache tier.
• Preserve the existing GPU/host-tier behavior on discrete GPU systems.

Root cause

KV cache sizing does not distinguish discrete GPUs with separate host DRAM from integrated GPUs with physically unified CPU/GPU memory. The correct CUDA signal for this case is cudaDevAttrIntegrated; cudaDevAttrPageableMemoryAccess is not sufficient because it can also be true on HMM-enabled discrete GPU systems.

Proposed / validated fix

PR #12301 implements the fix direction:

• Add integrated-device detection using cudaDevAttrIntegrated.
• Add Python-side KvCacheConfig auto-detection via is_device_integrated().
• Force host_cache_size to 0 when unified memory is detected.
• Add a C++ defensive guard in calculateFreeMemBytes() so C++ executor users that bypass Python validation still get freeSecondaryMemBytes = 0 on unified-memory systems.

Test evidence from PR context

• Verified is_device_integrated() returns true on DGX Spark GB10 / SM121 with about 121.7 GB visible memory.
• Verified the KV-cache config auto-detection path on DGX Spark.
• Pre-commit checks pass according to the PR context.
• Full KV-cache unit and inference workloads are deferred to NVIDIA CI because the local environment does not have the required internal model data / weights.

Related PR

• #12301

Review PR #12301 and trace the Python KvCacheConfig auto-detection path through is_device_integrated(), then inspect the C++ calculateFreeMemBytes() guard. Confirm integrated devices force host_cache_size and freeSecondaryMemBytes to zero while discrete GPU behavior remains unchanged. Run the available pre-commit and KV-cache tests; full workloads require NVIDIA CI and internal model data.