Advice for running RT-1 in a simple Pick and Place environment

Hi,
first of all, thank you for open-sourcing RT-1, collating the Open X-embodiment dataset, and releasing trained RT-1 and RT-1-X checkpoints. We are very impressed by the reported capabilities and eager to build on top of this.
We have been trying to use the model on a Franka in our office but moved to a minimalistic pybullet environment for easier experimentation since we are struggling to replicate reasonable behaviors for simple tasks. We tried finding a camera frame and a world frame in which interpreting the action as a position delta seems reasonable, but to no avail.

Our question is this: Do you have advice based on your own experiments for running inference on out-of-distribution settings, particularly regarding the following decisions:

- What is a recommended coordinate frame to interpret the delta positions? Which direction should be X, Y and Z?

- We observed that the model behaves differently when choosing different camera positions. What camera position could you recommend for the beginning?

- What is a reasonable starting point for denormalization of the actions for the Franka Panda robot? Currently we use the same values as in the bridge dataset.

Is it expected that one would first need to do some fine-tuning to align the model to a particular action space in an unseen setting? Note that we currently only try to pick up a simple cube, which, we believe, should be within the model's capabilities.
Kind regards,
Felix