allenai / allenai/ai2thor

Noise in RGB Images During Imitation Learning

Offen
#1,247 1 Kommentar 1 Reaktion 0 zugewiesene Personen Auf GitHub ansehen
Vorherrschende Sprache
C#
Sterne
1.8k
Forks
296
PR-Merge-Kennzahlen
Keine gemergten PRs in 30 T.

Beschreibung

Hello,

I am currently using your code for imitation learning and have encountered an issue where the RGB images occasionally contain noise. This occurs while I am saving the RGB images captured by the robot.

**Environment Configuration:**

STRETCH_ENV_ARGS = dict(
gridSize=AGENT_MOVEMENT_CONSTANT
* 0.75, # Intentionally make this smaller than AGENT_MOVEMENT_CONSTANT to improve fidelity
width=INTEL_CAMERA_WIDTH,
height=INTEL_CAMERA_HEIGHT,
visibilityDistance=MAXIMUM_DISTANCE_ARM_FROM_AGENT_CENTER,
visibilityScheme="Distance",
fieldOfView=INTEL_VERTICAL_FOV,
server_class=ai2thor.fifo_server.FifoServer,
useMassThreshold=True,
massThreshold=10,
autoSimulation=False,
autoSyncTransforms=True,
renderInstanceSegmentation= True,
agentMode="stretch",
renderDepthImage=SAVE_DEPTH,
cameraNearPlane=0.01, # VERY VERY IMPORTANT
branch=None, # IMPORTANT do not use branch
commit_id=STRETCH_COMMIT_ID,
server_timeout=MAXIMUM_SERVER_TIMEOUT,
snapToGrid=False,
fastActionEmit=True,
action_hook_runner=_ACTION_HOOK_RUNNER,
# antiAliasing="smaa", # We can get nicer looking videos if we turn on antiAliasing and change the quality
# quality="Ultra",
)

controller_args={
**STRETCH_ENV_ARGS,
"platform": (
ai2thor.platform.OSXIntel64
if sys.platform.lower() == "darwin"
else ai2thor.platform.CloudRendering
),
}
controller = Controller(**controller_args)

**Command to Read RGB Images:**

event = controller.step("ToggleMagnetVisibility", visible=False, raise_for_failure=True, renderImageSynthesis=True)
static_rgb = event.frame # 224 396 3
gripper_rgb = event.third_party_camera_frames[0][:, :, :3] # 224 396 3

Could you please provide guidance on why this might be happening and how to resolve it? Any insights or suggestions would be greatly appreciated.

Thank you for your help!

Beitragsleitfaden

Beitragsleitfaden öffnen

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.