OpenGVLab / OpenGVLab/Ask-Anything
Egoschema results iconsistency
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 3.4k
- Forks
- 268
- PR merge metrics
- No merged PRs in 30d
Description
Hi, I've ran your code "demo_mistral.ipynb" with the exact same library and python versions yet my accuracy is around 0.5 as opposed to 63.6 from your paper. I'm using mistral, stage3 and the exact same folders that I downloaded of config_mistral.json. What am I doing wrong?
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with demo_mistral.ipynb and config_mistral.json, using the reported library and Python versions and the downloaded folders. Compare the observed accuracy of about 0.5 with the reported 63.6 result and record the exact setup and output. Done means identifying the setup discrepancy or documenting why the result cannot be reproduced.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- jupyter-notebook, python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100