PEtab-dev / PEtab-dev/petab_select
Handle similar model criterion values
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 14
- Forks
- 1
- PR merge metrics
- No merged PRs in 30d
Description
Models can have indistinguishable criterion values, such that choosing one over the other is determined by numerical noise.
For example, in one case, during a forward search, two models with the same number of parameters were calibrated to have the same likelihood to several decimal places. Repeating the forward search 100 times resulted in 2 different trajectories through model space, occurring approximately 50/50. This is because numerical noise determined which of the two similar models were chosen at this point during the forward search.
Options to handle:
- emit a warning that it is unclear which model to select, when models are very similar
- create branches in the model selection, when encountering models that are within some epsilon criterion of each other or the best model so far
- allow users to restart model selection at specific points, to explore trajectories that might have been chosen given different numerical noise, or were within some epsilon criterion of the best model so far
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No file, test, or entry point is named. Start by locating the forward-search and model-selection implementation, then clarify which handling option is wanted; done requires an agreed behavior for near-tied criterion values and coverage for the resulting trajectories or warning.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100