Qwen3-VL-235B-A22B SFT OOM issue

I'm using 8×8 A100 GPUs to fine-tune (SFT) the Qwen3-VL-235B-A22B-Instruct model, but I keep encountering out-of-memory (OOM) issues regardless of the settings. Could you please advise me on how to resolve this? I’ve set IMAGE_MAX_TOKEN_NUM to 256, and each query contains at most two images. Based on my estimate, the total number of prompt tokens—including image tokens—should not exceed 1024, given the 256-token limit per image. Thanks

#! GQA limit： tensor_model_parallel_size=4
#sft command
PYTORCH_CUDA_ALLOC_CONF='expandable_segments:True'
OMP_NUM_THREADS=14
CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7
NNODES=8
NPROC_PER_NODE=8
NODE_RANK=$NODE_RANK
MASTER_ADDR=$MASTER_ADDR
MASTER_PORT=$MASTER_PORT
IMAGE_MAX_TOKEN_NUM=256
megatron sft
--load $MODEL_PATH
--dataset $DATA_PATH
--save $OUTPUT_DIR
--save_interval 300
--max_epochs 10
--tensor_model_parallel_size 4
--pipeline_model_parallel_size 2
--expert_model_parallel_size 8
--expert_tensor_parallel_size 1
--micro_batch_size 1
--global_batch_size 16
--sequence_parallel true
--moe_expert_capacity_factor 1
--moe_permute_fusion true
--moe_grouped_gemm true
--moe_shared_expert_overlap true
--recompute_granularity full
--recompute_method uniform
--recompute_num_layers 1
--finetune true
--cross_entropy_loss_fusion true
--lr 1e-5
--lr_warmup_fraction 0.05
--min_lr 1e-7
--moe_aux_loss_coeff 1e-6
--max_length 1024
--attention_backend flash
--optimizer_cpu_offload true
--optimizer_offload_fraction 1
--use_precision_aware_optimizer true
--packing true
--attn_impl flash_attn
--bf16 true
--num_workers 8
--no_save_optim true
--no_save_rng true
--dataset_num_proc 8
--freeze_vit true
--freeze_llm false
--freeze_aligner false
--split_dataset_ratio 0
--log_interval 1

Start by reviewing the supplied Megatron SFT command and its parallelism, sequence-length, recomputation, optimizer-offload, and image-token settings. Reproduce the OOM with the stated 8×8 A100 configuration, then determine which documented configuration change prevents the failure while preserving the requested fine-tuning setup.