openvinotoolkit / openvinotoolkit/physicalai

Documentation: PolicyRuntime in Runtime API Reference

Open
#176 0 comments 0 reactions 1 assignee View on GitHub

@samet-akcay is already working on this.

Since Jun 30, 2026.

Dominant language
Python
Stars
28
Forks
22
Avg merge
1d 17h
Merged PRs (30d)
30

Description

The document doesn't mention, but 'task' seems necessary.

When I run the script,

from physicalai.capture import UVCCamera
from physicalai.capture import RealSenseCamera
from physicalai.inference import InferenceModel
from physicalai.runtime import PolicyRuntime, SyncExecution
from physicalai.robot import SO101

#policy = InferenceModel("./SmolVLA/exports/openvino", device="GPU")
policy = InferenceModel("./SmolVLA/exports/openvino")

cameras = {
#    "bird view": UVCCamera(device="/dev/video32", width=960, height=720, fps=30), # v4l2-ctl --list-devices
    "bird view": UVCCamera(device="/dev/video40", width=1280, height=720, fps=30), # v4l2-ctl --list-devices
#    "side view": RealSenseCamera(serial_number="013222072639", width=640, height=480, fps=30),
}

robot = SO101(port="/dev/ttyACM1", calibration="./Bob.json", role="follower")
#robot.connect()

runtime = PolicyRuntime(
    fps=30,
    robot=robot,
    model=policy,
    cameras=cameras,
    execution=SyncExecution(),
#    task='pick pen and put it on cup',
)

with runtime:
    runtime.run(duration_s=60)

I got the error below.

Traceback (most recent call last):
File "/home/intel/openvino-physical-ai/test_inference.py", line 29, in
runtime.run(duration_s=60)
File "/home/intel/openvino-physical-ai/.venv/lib/python3.12/site-packages/physicalai/runtime/runtime.py", line 313, in run
self._warmup_with_retry()
File "/home/intel/openvino-physical-ai/.venv/lib/python3.12/site-packages/physicalai/runtime/runtime.py", line 590, in _warmup_with_retry
self._execution.warmup(sample_obs)
File "/home/intel/openvino-physical-ai/.venv/lib/python3.12/site-packages/physicalai/runtime/execution.py", line 107, in warmup
actions = self._model.predict_action_chunk(sample_observation)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/intel/openvino-physical-ai/.venv/lib/python3.12/site-packages/physicalai/inference/model.py", line 253, in predict_action_chunk
outputs = self(observation)
^^^^^^^^^^^^^^^^^
File "/home/intel/openvino-physical-ai/.venv/lib/python3.12/site-packages/physicalai/inference/model.py", line 203, in call
inputs = preprocessor(inputs)
^^^^^^^^^^^^^^^^^^^^
File "/home/intel/openvino-physical-ai/.venv/lib/python3.12/site-packages/physicalai/inference/preprocessors/new_line.py", line 33, in call
batch_tasks = inputs[TASK]
~~~~~~^^^^^^

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.