Megatron SFT full finetune breaks tool calling (refuses + hallucinates) while LoRA preserves it — Qwen3.6-27B on Ascend 910B3

Environment

• Hardware: Ascend 910B3 × 16 (2 nodes × 8 NPUs, multi-node)
• Model: Qwen3.6-27B (local weights, path omitted)
• Training: swift megatron sft, --agent_template qwen3_5
• Dataset: messages format JSONL (system / user / assistant), max_length 3000, packing false, padding_free false, domain QA SFT data (samples omitted for privacy)

Problem

Full finetune (LR 1e-5, global_bs 8, TP=4 PP=2, 5 epochs, save per epoch):

• Epoch 1 (loss ≈ 1.2): refuses to call tools, produces hallucinated / nonsensical output.
• Epoch 3 (loss ≈ 0.1): fully fits (memorizes) the training answers, but still refuses to call tools.

LoRA (rank 8, alpha 32, all-linear, LR 1e-4, TP=2 PP=1, same dataset):

• Epoch 3 (final loss ≈ 0.7): cannot fit the answers, but tool calling works fine.
• Epoch 5 (final loss ≈ 0.1): answers match the training set on ~50% of questions, tool calling works fine.

So full finetune fits the data but destroys tool calling; LoRA preserves tool calling but cannot fit the data.

Key args used (both runs)

megatron sft \
  --save_safetensors true --load_from_cache_file false \
  --freeze_llm false --finetune true \
  --tensor_model_parallel_size 4 --sequence_parallel true --pipeline_model_parallel_size 2 \  # full FT; LoRA: tp=2 pp=1
  --micro_batch_size 1 --global_batch_size 8 \
  --recompute_granularity full --recompute_method uniform --recompute_num_layers 1 \
  --cross_entropy_loss_fusion true \
  --lr 1e-5 --min_lr 1e-6 --lr_warmup_fraction 0.05 \  # LoRA: lr=1e-4 min_lr=1e-5
  --max_length 3000 --packing false --padding_free false \
  --save_strategy epoch --no_save_optim true --no_save_rng true \
  --agent_template qwen3_5
# LoRA run adds: --tuner_type lora --lora_rank 8 --lora_alpha 32 --target_modules all-linear --merge_lora true

Questions

1. Why would full-parameter Megatron SFT destroy tool calling within 1 epoch (refusal + hallucination), while LoRA on the same data keeps it intact? Is this a known issue with agent_template qwen3_5 under Megatron (e.g. tool-call token loss masking, chat template rendering differences, cross_entropy_loss_fusion / SP / PP interaction)?
2. Why can LoRA (rank 8, lr 1e-4) not fully fit the answers even at final loss ≈ 0.1, while full finetune can?
3. Any recommended settings to do full finetune while preserving tool-calling ability?

Any hints appreciated. Thanks!

Start at the swift megatron sft entry point and compare the full-finetune and LoRA configurations, especially --agent_template qwen3_5, loss settings, and TP/PP options. Reproduce the contrasting tool-calling behavior and isolate whether template rendering, token-loss handling, or parallelism is responsible; done means a confirmed cause and settings that preserve tool calling during full finetuning.