Evaluation script for mind2web?
Open
- Dominant language
- Python
- Stars
- 11.5k
- Forks
- 877
- PR merge metrics
- No merged PRs in 30d
Description
I wonder if there is any local evaluation script that includes data preprocessing and parsing of the outputs?
I was trying to create a similar one, but was unable to reproduce the results reported here.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by checking the repository for any existing Mind2Web evaluation entry points, data preprocessing, or output-parsing utilities. Compare those paths with the reported Mind2Web results and determine what is missing. Done means a local evaluation script can reproduce the reported results from the available data and model outputs.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100