The ib-irm CLI option silently dispatches to BCE
- Dominant language
- Python
- Stars
- 1
- Forks
- 1
- PR merge metrics
- No merged PRs in 30d
Description
## Summary
The CLI accepts the loss name `ib-irm`, but the training dispatcher checks for `ib_irm`. As a result, every `--loss ib-irm` run falls through to the default BCE branch instead of calling `ib_irm_loss`.
This is still present on `main` at `a0b47382f08b108a17a39b343c4e58dbf2852db3`.
## Evidence
`trial_training/train_plugins_v2.py` defines:
```python
parser.add_argument(... choices=[..., 'ib-irm', ...])
```
and `trial_training/run.sh` launches both May and June experiments with `--loss 'ib-irm'`.
However, the dispatch condition at line 108 is:
```python
elif loss_type == 'ib_irm':
```
The subsequent `else` at line 114 calls `bce_loss`, so the accepted CLI spelling can never reach the intended branch. The checkpoint naming code does use `ib-irm`, confirming that the hyphenated spelling is the public option.
## Minimal reproduction
Call `train_one_epoch(..., loss_type='ib-irm')` with `ib_irm_loss` replaced by a sentinel and `bce_loss` replaced by a recording stub. The BCE stub is called and the IB-IRM sentinel is not.
## Expected behavior
The accepted `ib-irm` option should dispatch to `ib_irm_loss` and incorporate both the IRM and information-bottleneck penalties.
## Impact
The IB-IRM experiments in `trial_training/run.sh` are labeled and checkpointed as IB-IRM while actually training with BCE (plus the shared L2 term), invalidating comparisons for that option.
## Suggested fix
Use the same canonical spelling in argument choices, dispatch, checkpoint naming, and tests. A focused regression test should assert that `ib-irm` invokes `ib_irm_loss` rather than the BCE fallback.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start in trial_training/train_plugins_v2.py at the loss argument and dispatcher around line 108, then check trial_training/run.sh for the public spelling. Add a focused regression test for train_one_epoch using the issue’s sentinel and recording-stub setup; done means `ib-irm` calls ib_irm_loss rather than the BCE fallback and the relevant tests pass.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- cli, machine-learning
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 88/100