google-deepmind / google-deepmind/mujoco

[Unity plugin] 2 tests stochastically fail in the plugin test suite.

Open
#3,175 1 comment 0 reactions 1 assignee Claimed by @Balint-H View on GitHub
bug Unity
Dominant language
C++
Stars
15.2k
Forks
1.8k
Avg merge
10d 16h
Merged PRs (30d)
25

Description

### Intro

During the maintenance of the Unity plugin I ran into strange behaviour in the test suite.

### My setup

This was observed on Windows 11, Unity 6000.3.10f1, and using #3164 version of the plugin.

### What's happening? What did you expect?

Two of the tests of the Unity plugin, namely `MjcfHierarchyMirrorsTheHierarchyOfUnityObjects` and `SceneRecreatedAfterDeletion` fail stochastically (i.e. rerunning the tests causes them to pass sometimes).

It is unclear at the moment whether it is a limitation of the tests or if it is a bug, but warrants some attention.

The first suspicion is that it might be due to the ordering of components not being deterministic, but in an initial scan I could not find a step that is known to have non-defined orders.

### Steps for reproduction

Add the plugin from disk to a Unity project, then add `"testables": ["org.mujoco"]` to the project manifest. Afterwards, you can run the tests from Window>General>Test runner. Running the tests multiple times will cause the tests to sometimes fail or succeed, seemingly randomly.

### Minimal model for reproduction

_No response_

### Code required for reproduction

_No response_

### Confirmations

- [x] I searched the [latest documentation](https://mujoco.readthedocs.io/en/latest/overview.html) thoroughly before posting.
- [x] I searched previous [Issues](https://github.com/google-deepmind/mujoco/issues) and [Discussions](https://github.com/google-deepmind/mujoco/discussions), I am certain this has not been raised before.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.