google-deepmind / google-deepmind/dm_control

[Question] canonical way to implement synchronous actions?

Open
#361 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
4.7k
Forks
764
PR merge metrics
No merged PRs in 30d

Description

I'm interested in using `mujoco` and `dm_control` to simulate a robot that can execute a trajectory for each step in the environment (e.g. for simulating a motion primitive with spatial actions).

In pybullet (or mujoco w/o the dm_control framework) in `environment.step` you would do IK, set the joint targets and call `p.stepSimulation()` untill the target pose is reached according to some criterion or just teleoport the joints to this state, collect the observations and return. But in dm_control, you cannot do this as the callbacks should be executed in time (so that other entities could also update their state based on the physics state or the action).

The number of physics steps required to execute this trajectory can vary quite a bit (nearby pose vs far away, or even teleporting the robot if it is not interacting with objects at that moment), so just running the worst case number of physics steps for each environment step seems very inefficient.

I'm wondering if this kind of synchronous (= "runtime-validated stop condition and hence variable number of physics steps") actions can be realised more efficiently in the `dm_control` framework, and what the canonical way is to do so.

One way I can think of, is an 'early termination' in the inner loop in the `environment.step` method, if the task signals this to the environment (the task itself can query its entities if needed), but that is not facilitated by the current Environment and I feel like I can't be the first person with this question so I wanted to hear if anyone knows a way to do this within the current framework?

Contributor guide

Open the contributing guide

Research direction

Start with the current dm_control Environment and its environment.step implementation, then inspect how task callbacks are scheduled during physics steps. Compare that flow with the proposed early-termination condition and the MuJoCo or dm_control trajectory use case. Done would be a documented canonical approach or a clearly scoped framework change for variable-length synchronous actions.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
robotics
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.