automl / automl/RNAformer

Unable to reproduce results on bpRNA

Open
#8 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
43
Forks
10
PR merge metrics
No merged PRs in 30d

Description

I attempted to reproduce your results on the bpRNA dataset by following the instructions in the README file.
However I got results that are different than the published ones.

# My Results
| solved | accuracy | precision | recall | f1_score | mcc | num_samples | test_set |
-----------:|------------:|------------:|----------:|-----------:|---------:|--------------:|:-----------|
| 0.00229885 | 0.000382429 | 0.304632 | 0.0637105 | 0.0917703 | 0.121859 | 1305 | bprna_ts0 |

# Expected Results (from the paper)
| solved | f1_score | mcc |
|----------|:-------------:|------:|
| 17.2% | 0.728 | 0.733 |

Perhaps I did something wrong during the setup?

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the README's bpRNA reproduction instructions and compare the setup and evaluation steps with the reported run on test set bprna_ts0. Confirm the discrepancy against the paper's expected results, then document the cause or the missing setup step and provide a reproducible result.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
bioinformatics, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.