OpenGVLab / OpenGVLab/InternVideo

[InternVideo1-B] Evaluation Not Triggered During Training with Specified Interval

Open
#225 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
2.4k
Forks
160
PR merge metrics
No merged PRs in 30d

Description

Dear InternVideo Group,

Thank you for your wonderful work!

I encountered an issue while attempting to finetune the InternVideo1-B model. According to the config instructions, I set the 'evaluation' as follows:

evaluation = dict( interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'], topk=(1, 5) )

This setting should trigger evaluation after every 5 epochs during training. However, based on my logs, it appears that the evaluations are not being conducted as expected. Specifically, the log output indicates that only the training process is taking place, with no records of evaluation activities.

Could you please advise if there's anything incorrect in my config? Additionally, any guidance on what might be causing this discrepancy and how to resolve it would be greatly appreciated.

Thank you in advance for your help.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by locating the finetuning entry point and the configuration containing evaluation = dict(interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'], topk=(1, 5)). Check how the training loop consumes this setting and compare the run logs with the expected evaluation schedule. Done means evaluation runs every five epochs and records the requested metrics.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.