microsoft / microsoft/Graphormer
Evaluation gets nan mae
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 2.5k
- Forks
- 374
- PR merge metrics
- No merged PRs in 30d
Description
I want to evaluate the performance of pretrained Graphormer using testing data and I use following code
python evaluate.py \
--user-dir ../../graphormer \
--num-workers 16 \
--ddp-backend=legacy_ddp \
--dataset-name pcqm4m \
--dataset-source ogb \
--task graph_prediction \
--criterion l1_loss \
--arch graphormer_base \
--num-classes 1 \
--batch-size 64 \
--pretrained-model-name pcqm4mv1_graphormer_base \
--load-pretrained-model-output-layer \
--split test \
--seed 1
It is similar to code in https://graphormer.readthedocs.io/en/latest/Quick-Start.html#evaluate-pre-trained-models, and the only difference is --split.
However, I get result like this. I don't know why.

Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with evaluate.py and the documented Quick-Start evaluation command, then compare the effects of using --split test instead of the documented split. Run the provided command and trace where the MAE is computed or reported; done means identifying why the test evaluation produces nan and documenting or correcting that behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100