google-deepmind / google-deepmind/mujoco_mpc
Question for a Feature: How to Expose Backward Pass Quantities?
- Dominant language
- C++
- Stars
- 1.7k
- Forks
- 282
- PR merge metrics
- No merged PRs in 30d
Description
Hello,
For context, I am a PhD student studying the interplay and intersection of MPC and RL. I was wondering what is the best way to go about exposing the iLQG value function and Q functions gradients and Hessians seen in `iLQGBackwardPass` in `backward_pass.h`? My end goal is to be able to call these values in the Python API agent.
Forgive my very limited understanding of how grpc works, but would the following idea be the general approach to doing this?
1. Extend the protocol buffer interface with messages like `GetBackwardPassResultsRequest` and `GetBackwardPassResultsResponse` in `agent.proto`.
2. Add the corresponding methods to `agent_service.h` and `agent_service.cc`.
3. Add getter functions in `backward_pass.h` and `backward_pass.cc` and expose them too in `planner.h` and `planner.cc`
4. Rebuild mjpc
5. Modify `agent.py` to have `get_backwardpass_results` methods that call a request and get a response from grpc.
I'm unsure of how the details would work on certain steps:
- For step 3, it would be more proper to add the getter functions to the `policy.h` and `policy.cc` instead of in the `planner`? My understanding (which might be wrong) is that we call an agent, which has a planner. The agent calls the planner to create optimal control gains, which is stored in the policy. The agent then calls the policy when it wants to apply those control gains.
- For step 4, would it suffice to just rerun `python setup.py install` as seen in the [Python API installation instructions](https://github.com/google-deepmind/mujoco_mpc?tab=readme-ov-file#install-api) to now have access to those values from the Python `agent`?
Thank you in advance for any help and guidance!
Contributor guide
Assessment
This issue has not been assessed yet.