OpenBMB / OpenBMB/MiniCPM-Robot

[Feature Request] Open-source training / fine-tuning scripts for MiniCPM-RobotManip

Open
#6 0 comments 8 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
333
Forks
29
Avg merge
19m
Merged PRs (30d)
1

Description

We are trying to adapt MiniCPM-RobotManip to a new robot arm for a specific manipulation task. The inference side is clear — vla_infer.py exposes the embodiment_id interface and action_head.max_num_embodiments for multi-embodiment support. However, the current repository only includes inference code; no training or fine-tuning scripts are provided for RobotManip.

Questions

  1. Fine-tuning scripts: Is there a plan to open-source the training/fine-tuning code for MiniCPM-RobotManip? Even a minimal script covering data loading, optimizer setup, and checkpoint saving would be very helpful.

  2. New embodiment registration: What is the recommended workflow for adding a new embodiment_id beyond the ones already in the released weights? Does this require retraining from scratch, or is there a fine-tuning path that only updates the action head for the new embodiment?

  3. Data format: Could you document or provide a sample of the training data format expected by RobotManip (state shape, action shape, image resolution, etc.)? The inference script implies STATE_DIM = 80 and action shape (30, 80), but the mapping from real robot joints to this 80-dim space is unclear.

  4. Data quantity guidance: Are there any ablation results or rule-of-thumb recommendations for how many demonstration episodes are needed to fine-tune on a new embodiment/task?

Any pointers — even a rough timeline or a link to an internal script — would be greatly appreciated. Thank you for the great work on this project!

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with vla_infer.py and its embodiment_id, action_head.max_num_embodiments, STATE_DIM=80, and (30, 80) action shape. A complete contribution would need to define training or fine-tuning entry points, new-embodiment registration, the expected data format, and guidance on demonstration quantity; no training files or tests are named in the issue.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning, robotics
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.