hans / hans/bayesian-erp

Model phoneme set is not cross-checked with stimulus phoneme set on reload

Open
#13 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
1
Forks
1
PR merge metrics
No merged PRs in 30d

Description

This can lead to catastrophic mismatches when a pickled model is reloaded with a stimulus representation whose phonemes have been reordered.

I think models should carry the original list of phonemes associated with the cognitive parameters (i.e. the confusion matrix) along with the confusion matrix itself. Then the representation can be cross-checked with the stimulus and rearranged if necessary.

Unfortunately we have a bunch of pickled model pipelines which don't have that metadata. So, two fixes necessary:
- [ ] ensure stimulus representation is backward-compatible by making sure phonemes come out in the same order as they used to (compare stimulus pre-syllable change and post-syllable change)
- [x] add metadata to model along with checks on deserializing from checkpoint. add tests.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.