Unity-Technologies / Unity-Technologies/ml-agents
CameraSensor NullReferenceException & Black Screen
Nobody has claimed this yet.
- Dominant language
- C#
- Stars
- 19.7k
- Forks
- 4.5k
- PR merge metrics
- No merged PRs in 30d
Description
Describe the bug
Whenever using a CameraSensor Component with a game object that has an Agent script, a NullReferenceException is raised and the camera that is tied to the sensor just shows a black screen with 'No cameras rendering'. Additionally, the corresponding ML-Agent Python scripts halts when connecting.
This appears to be a duplicate of issue #5961, but that was closed without resolution.
To Reproduce
Steps to reproduce the behavior:
- Open new project with Unity 'Universal 3D'
- Install the ML-Agents v4.1.0 package
- Add some game object, e.g. Cube
- Give the game object an Agent script
- Give the game object a CameraSensor component with default settings
- Assign the main camera to the CameraSensor
- Play the scene
Screenshots
Console logs / stack traces
In my main project where I was getting this problem, the console log I was getting was:
NullReferenceException: Object reference not set to an instance of an object
Unity.MLAgents.Sensors.CameraSensor.ObservationToTexture (UnityEngine.Camera obsCamera, UnityEngine.Texture2D texture2D, System.Int32 width, System.Int32 height) (at ./Library/PackageCache/com.unity.ml-agents@51fac9654958/Runtime/Sensors/CameraSensor.cs:159)
Unity.MLAgents.Sensors.CameraSensor.Update () (at ./Library/PackageCache/com.unity.ml-agents@51fac9654958/Runtime/Sensors/CameraSensor.cs:115)
Unity.MLAgents.Agent.UpdateSensors () (at ./Library/PackageCache/com.unity.ml-agents@51fac9654958/Runtime/Agent.cs:1154)
Unity.MLAgents.Agent.SendInfoToBrain () (at ./Library/PackageCache/com.unity.ml-agents@51fac9654958/Runtime/Agent.cs:1114)
Unity.MLAgents.Agent.SendInfo () (at ./Library/PackageCache/com.unity.ml-agents@51fac9654958/Runtime/Agent.cs:1377)
Unity.MLAgents.Academy.EnvironmentStep () (at ./Library/PackageCache/com.unity.ml-agents@51fac9654958/Runtime/Academy.cs:586)
Unity.MLAgents.AcademyFixedUpdateStepper.FixedUpdate () (at ./Library/PackageCache/com.unity.ml-agents@51fac9654958/Runtime/Academy.cs:43)
But in the reproduction environment described earlier, I was unable to produce any raised exception.
Environment (please complete the following information):
- Unity Version: 6000.3.6f1
- OS + version: Windows 10 & Debian 13
- ML-Agents version: ML-Agents v4.1.0
- Environment: Unity Universal 3D
NOTE: We are unable to help reproduce bugs with custom environments. Please attempt to reproduce your issue with one of the example environments, or provide a minimal patch to one of the environments needed to reproduce the issue.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the reproduction steps in a Unity Universal 3D project, then inspect Runtime/Sensors/CameraSensor.cs at the reported ObservationToTexture and Update lines, along with the Agent.cs sensor update path. Done means reproducing and explaining the NullReferenceException, black camera output, and Python connection halt, or documenting why the supplied environment does not reproduce them.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp, machine-learning, python, unity
- Domain
- game-dev, machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100