anthony-wang / anthony-wang/CrabNet

CrabNet matbench data, possible mismatch between submission notebook and results

Aperta
#23 2 commenti 0 reazioni 0 assegnatari Vedi su GitHub
Lingua principale
Python
Stelle
132
Fork
42
Metriche di merge delle PR
Nessuna PR unita negli ultimi 30g

Descrizione

@MahamadSalah74 I ran the [matbench notebook](https://github.com/materialsproject/matbench/blob/main/benchmarks/matbench_v0.1_CrabNet/notebook.ipynb) posted on the matbench GitHub page (https://github.com/materialsproject/matbench/pull/23), and got somewhat higher MAEs for a few repeat runs (see below) compared to what was reported.
```python
0.3683
0.3661
0.3658
```
The reported matbench result (`0.3463`) seems a bit more in line with `matbench_crabnet.py` which uses 300 epochs and the full train/val dataset for training. It's not a huge difference, but I'm trying to figure out if/what the discrepancy is.

Maybe I'm missing something basic. Could you comment on this?

Guida per i contributori

Nessuna guida per i contributori indicizzata per questo repository

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.