Farama-Foundation / Farama-Foundation/MiniWoB-plusplus
[Ongoing] Find & mark non-deterministic environments
- Dominant language
- HTML
- Stars
- 400
- Forks
- 59
- PR merge metrics
- No merged PRs in 30d
Description
HTML and JavaScript are never meant to be run deterministically (or even synchronously), same story with web browsers. As a result, web-based MiniWoB++ suffer from determinism issues. Some are due to bugs in script, others are sort of inevitable.
If you find any non-deterministic environments, please share it via replying to this issue, and we would investigate & mark the environment as a non-deterministic one. It would be helpful to provide observation dumps or any other useful debug info, but just having the name is enough.
Relevant pull requests:
- #116
- #118
### Checklist
- [x] I have checked that there is no similar [issue](https://github.com/Farama-Foundation/miniwob-plusplus/issues) in the repo (**required**)
Contributor guide
Research direction
Review the web-based MiniWoB++ environments and investigate whether their behavior is reproducible across runs. Share the environment name with observation dumps or other useful debug information; done means the issue has been investigated and the environment is marked as non-deterministic when appropriate.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- html, javascript
- Domain
- testing, web-dev
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100