Feature Request : Gradient Statistics Monitoring Callback

Description & Motivation

While PyTorch Lightning provides useful callbacks for monitoring learning rates, device statistics, and throughput, there is currently no built-in utility for tracking gradient behavior.

Having access to gradient statistics (e.g., norm, mean, variance, sparsity) can significantly help in debugging training instability, tune hyperparameters, and better understand model behavior.

Pitch

I propose adding a GradientStatsMonitor callback that logs gradient-related statistics during training.

The callback could:

• Compute global gradient norm across all parameters
• Optionally compute per-layer gradient norms
• Track basic statistics such as mean, standard deviation, and fraction of near-zero gradients
• Log metrics through the existing Lightning logger interface
• Optionally provide warnings for potential issues (e.g., exploding or vanishing gradients)

Alternatives

No response

Additional context

No response

cc @lantiga

Start by reviewing the existing callbacks for learning-rate, device-statistics, and throughput monitoring, then inspect the Lightning logger interface mentioned in the proposal. Define the scope for global and per-layer gradient statistics, optional warnings, and logging behavior before implementation; done means the callback supports the agreed metrics and integrates with the existing logger.