About the resullt in ogbg-molhiv

``Thanks for the code. Good job!

At present, I am trying do some work based on Graphormer. And I try to reproduce the result in ogbg-molhiv, but meet some problems.
I train the model in 2 x RTX 3090(24G), CUDA_VERSION:11.1, and the version of pytorch is same as the github project.

I train the model use this script

n_gpu=2
epoch=8
max_epoch=$((epoch + 1))
batch_size=64
tot_updates=$((33000*epoch/batch_size/n_gpu))
warmup_updates=$((tot_updates/10))

CUDA_VISIBLE_DEVICES=1,2 fairseq-train \
--user-dir graphormer \
--num-workers 16 \
--ddp-backend=legacy_ddp \
--dataset-name ogbg-molhiv \
--dataset-source ogb \
--task graph_prediction_with_flag \
--criterion binary_logloss_with_flag \
--arch graphormer_base \
--num-classes 1 \
--attention-dropout 0.1 --act-dropout 0.1 --dropout 0.0 \
--optimizer adam --adam-betas '(0.9, 0.999)' --adam-eps 1e-8 --clip-norm 5.0 --weight-decay 0.0 \
--lr-scheduler polynomial_decay --power 1 --warmup-updates $warmup_updates --total-num-update $tot_updates \
--lr 2e-4 --end-learning-rate 1e-9 \
--batch-size $batch_size \
--fp16 \
--data-buffer-size 20 \
--encoder-layers 12 \
--encoder-embed-dim 768 \
--encoder-ffn-embed-dim 768 \
--encoder-attention-heads 32 \
--max-epoch $max_epoch \
--save-dir $save_dir_root \
--pretrained-model-name pcqm4mv1_graphormer_base \
--seed ${seeds[$i]} \
--flag-m 3 \
--flag-step-size 0.001 \
--flag-mag 0.001 \
--tensorboard-logdir $tensorboard_dir_root \
--log-format simple --log-interval 100 \
--log-file $log_dir 

And evalute the model use:

CUDA_VISIBLE_DEVICES=3 python graphormer/evaluate/evaluate.py \
    --user-dir graphormer \
    --num-workers 16 \
    --ddp-backend=legacy_ddp \
    --dataset-name ogbg-molhiv \
    --dataset-source ogb \
    --task graph_prediction \
    --arch graphormer_base \
    --num-classes 1 \
    --batch-size $batch_size \
    --save-dir $save_dir_root  \
    --metric auc \
    --seed ${seeds[$i]} \
    --sfilename $result_dir \
    --log-format simple   

I use seeds 1-5 util now.
And the result is:
{'epoch-best': {'val': {'auc': 0.7915973390450101}, 'test': {'auc': 0.7689351341951192}}}(seed-1)
{'epoch-best': {'val': {'auc': 0.7967697158563377}, 'test': {'auc': 0.7800533302417252}}}(seed-2)
{'epoch-best': {'val': {'auc': 0.7556909933843831}, 'test': {'auc': 0.7775153131219446}}}(seed-3)
{'epoch-best': {'val': {'auc': 0.7953004299078593}, 'test': {'auc': 0.799790350317856}}}(seed-4)
{'epoch-best': {'val': {'auc': 0.7998829473968052}, 'test': {'auc': 0.7942418796977954}}}(seed-5)

And the results with pretrain model pcqm4mv2_graphormer_base are also not optimistic.
emmm, I don't know what happens.
Looking forward to your reply.

Start with the training command and graphormer/evaluate/evaluate.py, then compare the ogbg-molhiv configuration, seeds, pretrained model, and reported AUC values. Check whether the documented training and evaluation setup reproduces the expected results; done means identifying the source of the discrepancy or documenting a reproducible explanation.