Reject log encoding on a categorically-sampled env_param: the combination is contradictory and degenerates in practice
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 99
- Forks
- 62
- Avg merge
- 6d 12h
- Merged PRs (30d)
- 17
Description
Requirement
A parameter declared as categorical must not be log-encodable. The combination should be rejected at validation rather than accepted.
Declaring a discrete candidate set and then presenting it to the policy as a continuous magnitude is contradictory on its face, and it is confusing to read in a config.
Why this is not an edge case
Core has exactly one sampling mode: an env_param draws from a candidate list in cmd_args.<name>, resolved by EnvParams.from_test. There is no continuous sampling.
So every env_param is categorically sampled, by construction. Which means any non-categorical encoding can only ever be applied to a categorically-sampled parameter — the very combination this issue says should be disallowed.
That has a direct consequence for #1035, which I opened: as designed, LogEncoding has no legal use. Its only possible application is to a candidate list, so either
- it is withdrawn, or
- it is paired with a continuous sampling mode, so a log-encoded parameter is one that was genuinely drawn from a range.
Evidence that the combination is harmful, not just confusing
From a trained Stage-3 checkpoint's observation-filter state, for two log-encoded parameters (each [is_zero, log10(value)]):
dims 0-1 drop_rate mean 0.497, -3.000 std 0.500, 0.000
dims 2-3 msg_size mean 0.000, 3.922 std 0.000, 0.740
Two of those four dimensions are dead:
log10(drop_rate)is a constant −3.000, std exactly 0. The candidates were{0.0, 0.001}and the encoding's floor maps both to −3.0, so the log half carries no information whatsoever —is_zerodoes all the work, which is a binary indicator, which is what categorical would have produced anyway.msg_size'sis_zerois identically 0, since a message size is never zero.
So for a categorically-sampled parameter the log encoding degenerates: it spends two dimensions to convey what one carries, and in the drop_rate case conveys nothing at all in the log dim. The zero-variance dimension then has to be defended downstream by an epsilon guard in the divisor, for a dimension that could not have varied.
What the encoding actually buys, for the record
The one property worth keeping in mind when deciding: log width is fixed at 2 regardless of candidate count, whereas categorical width grows with the list. Adding a fourth msg_size candidate changes the observation width under categorical (breaking trained checkpoints loudly) but not under log (loading and extrapolating silently).
That is an argument about architecture stability, not about representing a discrete set as a magnitude — and it is the only argument that survives the degeneracy above. If that property is wanted it should be pursued deliberately, not obtained as a side effect of mislabelling a categorical parameter.
Suggested resolution
- Validate that
encodingis compatible with how the parameter is sampled; reject a non-categorical encoding on a candidate-list parameter with a message naming both halves. - Resolve #1035 accordingly — withdraw it, or extend it with the continuous sampling mode that would make a log encoding meaningful.
Raised while auditing an RL check-in whose configs used sampling = { type = "categorical", ... } together with encoding = { type = "log" } on the same parameter.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Begin with EnvParams.from_test and trace the validation path for env_param encodings and candidate lists. Check how categorical sampling and non-categorical encodings are represented, then identify the existing validation coverage before defining the rejection behavior. Done means a categorical candidate-list parameter cannot use a non-categorical encoding, with an error naming both settings; the relationship to #1035 should also be resolved.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100