NVIDIA / NVIDIA/cloudai

Reject log encoding on a categorically-sampled env_param: the combination is contradictory and degenerates in practice

Open
#1,037 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
99
Forks
62
Avg merge
6d 12h
Merged PRs (30d)
17

Description

Requirement

A parameter declared as categorical must not be log-encodable. The combination should be rejected at validation rather than accepted.

Declaring a discrete candidate set and then presenting it to the policy as a continuous magnitude is contradictory on its face, and it is confusing to read in a config.

Why this is not an edge case

Core has exactly one sampling mode: an env_param draws from a candidate list in cmd_args.<name>, resolved by EnvParams.from_test. There is no continuous sampling.

So every env_param is categorically sampled, by construction. Which means any non-categorical encoding can only ever be applied to a categorically-sampled parameter — the very combination this issue says should be disallowed.

That has a direct consequence for #1035, which I opened: as designed, LogEncoding has no legal use. Its only possible application is to a candidate list, so either

  • it is withdrawn, or
  • it is paired with a continuous sampling mode, so a log-encoded parameter is one that was genuinely drawn from a range.

Evidence that the combination is harmful, not just confusing

From a trained Stage-3 checkpoint's observation-filter state, for two log-encoded parameters (each [is_zero, log10(value)]):

dims 0-1  drop_rate   mean  0.497, -3.000    std  0.500, 0.000
dims 2-3  msg_size    mean  0.000,  3.922    std  0.000, 0.740

Two of those four dimensions are dead:

  • log10(drop_rate) is a constant −3.000, std exactly 0. The candidates were {0.0, 0.001} and the encoding's floor maps both to −3.0, so the log half carries no information whatsoever — is_zero does all the work, which is a binary indicator, which is what categorical would have produced anyway.
  • msg_size's is_zero is identically 0, since a message size is never zero.

So for a categorically-sampled parameter the log encoding degenerates: it spends two dimensions to convey what one carries, and in the drop_rate case conveys nothing at all in the log dim. The zero-variance dimension then has to be defended downstream by an epsilon guard in the divisor, for a dimension that could not have varied.

What the encoding actually buys, for the record

The one property worth keeping in mind when deciding: log width is fixed at 2 regardless of candidate count, whereas categorical width grows with the list. Adding a fourth msg_size candidate changes the observation width under categorical (breaking trained checkpoints loudly) but not under log (loading and extrapolating silently).

That is an argument about architecture stability, not about representing a discrete set as a magnitude — and it is the only argument that survives the degeneracy above. If that property is wanted it should be pursued deliberately, not obtained as a side effect of mislabelling a categorical parameter.

Suggested resolution

  1. Validate that encoding is compatible with how the parameter is sampled; reject a non-categorical encoding on a candidate-list parameter with a message naming both halves.
  2. Resolve #1035 accordingly — withdraw it, or extend it with the continuous sampling mode that would make a log encoding meaningful.

Raised while auditing an RL check-in whose configs used sampling = { type = "categorical", ... } together with encoding = { type = "log" } on the same parameter.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Begin with EnvParams.from_test and trace the validation path for env_param encodings and candidate lists. Check how categorical sampling and non-categorical encodings are represented, then identify the existing validation coverage before defining the rejection behavior. Done means a categorical candidate-list parameter cannot use a non-categorical encoding, with an error naming both settings; the relationship to #1035 should also be resolved.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.