alibaba / alibaba/AliceMind

Grounding checkpoint evaluation results

Open
#82 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
2k
Forks
301
PR merge metrics
No merged PRs in 30d

Description

The checkpoint file is "output/vg_large_uni/checkpoint_best.pth" in grounding_mplug_large.sh. Is it same with "mplug_large_v2.pth"? I evaluate refcoco and the accuracy is only 0.02. Could you provide the grounding model? Thanks in advance.

`# eval refcoco
CUDA_VISIBLE_DEVICES=0 python -m torch.distributed.launch --nproc_per_node=1 --master_port=3222 --use_env grounding_mplug.py \
--config configs/grounding_mplug_large.yaml \
--dataset vg_unc \
--output_dir ./output/vg_large_unc \
--eval_checkpoint mplug_large_v2.pth \
--do_two_optim --evaluate`

{'test_model:': 'mplug_large.pth', 'testA_set_accuracy': 0.020682340463143007}

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with grounding_mplug_large.sh and the grounding_mplug.py command shown in the issue, then inspect configs/grounding_mplug_large.yaml and the referenced checkpoint names. Reproduce the RefCOCO evaluation and determine whether the checkpoint used by the script matches the reported model; done means the checkpoint relationship and the cause of the 0.02 accuracy are documented.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.