google-deepmind / google-deepmind/mujoco
[Unity plugin] 2 tests stochastically fail in the plugin test suite.
- Dominant language
- C++
- Stars
- 15.2k
- Forks
- 1.8k
- Avg merge
- 10d 16h
- Merged PRs (30d)
- 25
Description
### Intro
During the maintenance of the Unity plugin I ran into strange behaviour in the test suite.
### My setup
This was observed on Windows 11, Unity 6000.3.10f1, and using #3164 version of the plugin.
### What's happening? What did you expect?
Two of the tests of the Unity plugin, namely `MjcfHierarchyMirrorsTheHierarchyOfUnityObjects` and `SceneRecreatedAfterDeletion` fail stochastically (i.e. rerunning the tests causes them to pass sometimes).
It is unclear at the moment whether it is a limitation of the tests or if it is a bug, but warrants some attention.
The first suspicion is that it might be due to the ordering of components not being deterministic, but in an initial scan I could not find a step that is known to have non-defined orders.
### Steps for reproduction
Add the plugin from disk to a Unity project, then add `"testables": ["org.mujoco"]` to the project manifest. Afterwards, you can run the tests from Window>General>Test runner. Running the tests multiple times will cause the tests to sometimes fail or succeed, seemingly randomly.
### Minimal model for reproduction
_No response_
### Code required for reproduction
_No response_
### Confirmations
- [x] I searched the [latest documentation](https://mujoco.readthedocs.io/en/latest/overview.html) thoroughly before posting.
- [x] I searched previous [Issues](https://github.com/google-deepmind/mujoco/issues) and [Discussions](https://github.com/google-deepmind/mujoco/discussions), I am certain this has not been raised before.
Contributor guide
Assessment
This issue has not been assessed yet.