LLM Post training - Question regarding seq-mask-tis and global_valid_toks denominator scaling.

Reference: https://github.com/mlcommons/training/tree/master/llm_post_training

While looking at the GRPO loss implementation, I ran into something with the seq-mask-tis filter that I wanted to verify.

It looks like when truncated_importance_sampling_type == "seq-mask-tis", sequences that fall outside the band have their importance weights zeroed out, but those dropped sequences are still kept in the global_valid_toks denominator during the final loss reduction

Here is the section in ClippedPGLossFn.call where the mask is applied to the weights:

elif self.truncated_importance_sampling_type == "seq-mask-tis":
                # ... [snip] ...
                seq_kept_mask = (
                    (
                        seq_geomean_is_ratio
                        >= self.truncated_importance_sampling_ratio_min
                    )
                    & (seq_geomean_is_ratio <= self.truncated_importance_sampling_ratio)
                ).float()  # [B]
                # ...
                actor_importance_weights_expanded = (
                    actor_importance_weights_expanded * seq_kept_mask.unsqueeze(-1)
                )

And then immediately below, the loss is reduced using the upstream global_valid_toks:

if self.loss_type == LossType.TOKEN_LEVEL:
            actor_loss = masked_mean(
                importance_weights_to_use * clip_loss,
                mask,
                global_normalization_factor=global_valid_toks,
            )

Because global_valid_toks is calculated upstream (before this loss function zeroes out the actor_importance_weights_expanded), the dropped sequences are still acting as a denominator.

If the filter drops 30% of the sequences, isn't the overall gradient silently scaled down to 70% of nominal, rather than acting as a true average over just the surviving tokens.

I noticed that other sequence-dropping mechanisms (like the overlong filter) avoid this by zeroing out the sample_mask before global_valid_toks is calculated, which correctly removes them from the denominator.

Is leaving the seq-mask-tis drops in the denominator intentional here to act as a dynamic learning rate penalty, or should this be following the same convention as the overlong filter where dropped sequences are removed from the normalization factor?

Related: what is_oob_ratio does the reference typically see at the [0.999, 1.002] bounds? It's computed in _is_filter_metrics but doesn't appear in the published RCP logs, so it's hard to tell from outside how often this path is active.