anthony-wang / anthony-wang/CrabNet
CrabNet matbench data, possible mismatch between submission notebook and results
- Dominant language
- Python
- Stars
- 132
- Forks
- 42
- PR merge metrics
- No merged PRs in 30d
Description
@MahamadSalah74 I ran the [matbench notebook](https://github.com/materialsproject/matbench/blob/main/benchmarks/matbench_v0.1_CrabNet/notebook.ipynb) posted on the matbench GitHub page (https://github.com/materialsproject/matbench/pull/23), and got somewhat higher MAEs for a few repeat runs (see below) compared to what was reported.
```python
0.3683
0.3661
0.3658
```
The reported matbench result (`0.3463`) seems a bit more in line with `matbench_crabnet.py` which uses 300 epochs and the full train/val dataset for training. It's not a huge difference, but I'm trying to figure out if/what the discrepancy is.
Maybe I'm missing something basic. Could you comment on this?
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.