Model phoneme set is not cross-checked with stimulus phoneme set on reload
- Dominant language
- Jupyter Notebook
- Stars
- 1
- Forks
- 1
- PR merge metrics
- No merged PRs in 30d
Description
This can lead to catastrophic mismatches when a pickled model is reloaded with a stimulus representation whose phonemes have been reordered.
I think models should carry the original list of phonemes associated with the cognitive parameters (i.e. the confusion matrix) along with the confusion matrix itself. Then the representation can be cross-checked with the stimulus and rearranged if necessary.
Unfortunately we have a bunch of pickled model pipelines which don't have that metadata. So, two fixes necessary:
- [ ] ensure stimulus representation is backward-compatible by making sure phonemes come out in the same order as they used to (compare stimulus pre-syllable change and post-syllable change)
- [x] add metadata to model along with checks on deserializing from checkpoint. add tests.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.