Benchmarking prediction accuracy of different modes against input experimental structure.
Nobody has claimed this yet.
- Dominant language
- HTML
- Stars
- 115
- Forks
- 75
- PR merge metrics
- No merged PRs in 30d
Description
Description of feature
One use case for running multiple modes in parallel is to benchmark the accuracy of different methods when there is a known experimental structure for the prediction target.
It could be a nice option to enable users to input an experimental structure and score how well the predictions from each of the modes matches the provided experimental structure.
The boltz authors used OpenStructure to judge the quality of predictions for general molecules which would likely be an appropriate strategy here.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing how the workflow runs multiple prediction modes and how users could provide an experimental structure. Investigate OpenStructure as the proposed scoring approach. Done should include a documented way to compare each mode's predictions against the provided structure, with accuracy results that identify the best-performing mode.
Written by the indexing model from the issue text.
Assessment
- Domain
- machine-learning, testing-qa
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100