biorobotics / biorobotics/learning_modular_policies

Question About Propagation Logic Difference Between Training and Inference

Open
#3 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
4
Forks
0
PR merge metrics
No merged PRs in 30d

Description

Hi, thank you for open-sourcing this great project! I’ve learned a lot from the code, especially the structured modeling and control design.

When reading pgnn_control.py, I noticed something in the run_propagations function:
```
if not (last_step and module.my_gnn.output_len == 0):
module.aggregate_messages()
module.status_update()
```

This condition seems to skip aggregate_messages() and status_update() for some modules (e.g., leg or wheel modules) during policy inference in simulate_policy.py.

But in the training code mbrl.py, the run_propagations function always performs:
```
for module in modules_list:
module.aggregate_messages()
module.status_update()
```

So every module updates at every step.

My question:

From my understanding, all modules should also run aggregation and update during policy inference, otherwise some modules may not complete the intended propagation.

So I’m confused about:

Why does the control code skip propagation when (last_step and output_len == 0)?

Is this an intentional design difference between training and control?

Thanks for your time! I’d really appreciate any clarification on this behavior. 🙏

Contributor guide

No contributing guide indexed for this repository

Research direction

Compare run_propagations in pgnn_control.py and mbrl.py, then trace how simulate_policy.py invokes the control path. Determine whether the last_step and output_len condition is intentional; done means documenting the expected propagation behavior or identifying a concrete correction.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.