allenai / allenai/ai2thor

Noise in RGB Images During Imitation Learning

Abierto
#1,247 1 comentario 1 reacción 0 asignados Ver en GitHub
Lenguaje dominante
C#
Estrellas
1.8k
Forks
297
Métricas de merge de PR
Sin PR fusionados en 30 d

Descripción

Hello,

I am currently using your code for imitation learning and have encountered an issue where the RGB images occasionally contain noise. This occurs while I am saving the RGB images captured by the robot.

**Environment Configuration:**

STRETCH_ENV_ARGS = dict(
gridSize=AGENT_MOVEMENT_CONSTANT
* 0.75, # Intentionally make this smaller than AGENT_MOVEMENT_CONSTANT to improve fidelity
width=INTEL_CAMERA_WIDTH,
height=INTEL_CAMERA_HEIGHT,
visibilityDistance=MAXIMUM_DISTANCE_ARM_FROM_AGENT_CENTER,
visibilityScheme="Distance",
fieldOfView=INTEL_VERTICAL_FOV,
server_class=ai2thor.fifo_server.FifoServer,
useMassThreshold=True,
massThreshold=10,
autoSimulation=False,
autoSyncTransforms=True,
renderInstanceSegmentation= True,
agentMode="stretch",
renderDepthImage=SAVE_DEPTH,
cameraNearPlane=0.01, # VERY VERY IMPORTANT
branch=None, # IMPORTANT do not use branch
commit_id=STRETCH_COMMIT_ID,
server_timeout=MAXIMUM_SERVER_TIMEOUT,
snapToGrid=False,
fastActionEmit=True,
action_hook_runner=_ACTION_HOOK_RUNNER,
# antiAliasing="smaa", # We can get nicer looking videos if we turn on antiAliasing and change the quality
# quality="Ultra",
)

controller_args={
**STRETCH_ENV_ARGS,
"platform": (
ai2thor.platform.OSXIntel64
if sys.platform.lower() == "darwin"
else ai2thor.platform.CloudRendering
),
}
controller = Controller(**controller_args)

**Command to Read RGB Images:**

event = controller.step("ToggleMagnetVisibility", visible=False, raise_for_failure=True, renderImageSynthesis=True)
static_rgb = event.frame # 224 396 3
gripper_rgb = event.third_party_camera_frames[0][:, :, :3] # 224 396 3

Could you please provide guidance on why this might be happening and how to resolve it? Any insights or suggestions would be greatly appreciated.

Thank you for your help!

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.