Gradient reliability with sample-by-sample vs batch processing

I'm examining the training implementation in src/art/unsloth/service.py and have a question about the gradient computation approach.

Currently, the code processes samples individually:

for offset in range(0, packed_tensors["tokens"].shape[0]):
# Process single sample: v[offset : offset + 1]
# Each sample triggers separate gradient computation and parameter update

This means:

• Sample 1: θ₁ = θ₀ - lr * ∇L₁(θ₀)
• Sample 2: θ₂ = θ₁ - lr * ∇L₂(θ₁) (based on updated θ₁)
• Sample 3: θ₃ = θ₂ - lr * ∇L₃(θ₂) (based on updated θ₂)

Versus standard batch processing:

• All samples: θ = θ₀ - lr * (∇L₁(θ₀) + ∇L₂(θ₀) + ∇L₃(θ₀))/batch_size

Question:

What's the reasoning behind this sequential gradient approach? Does it provide better gradient reliability or learning dynamics for your specific use case?

I'm particularly curious whether this design choice stems from:

• Improved convergence properties
• Better handling of gradient variance
• Specific requirements for your training methodology

The downstream training code in train.py appears to support full batch processing, so I'm wondering if there are important gradient-related considerations I'm
missing.

Thanks for any insights!

Start with the sample-processing loop in src/art/unsloth/service.py and compare it with the batching path in train.py. Trace where gradients and parameter updates occur, then review the existing tests or training documentation for the intended behavior. Done means documenting the rationale for sequential processing, or identifying a concrete change if the behavior is unintended.