lyft / lyft/nuscenes-devkit

Some problems about sensor coord system

Open
#87 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Jupyter Notebook
Stars
390
Forks
99
PR merge metrics
No merged PRs in 30d

Description

I try like this:
```
my_sample = level5data.get('sample', my_sample_token)
sensor_channel_list=['CAM_FRONT', 'LIDAR_TOP', 'CAM_BACK']
for sensor_channel in sensor_channel_list:
my_sample_data = level5data.get('sample_data', my_sample['data'][sensor_channel])
Path, box_list, cam_intrinsic = level5data.get_sample_data(my_sample_data['token'])
print('{} channel path = {}'.format(sensor_channel, Path))
if any(box_list):
print('{} channel box = {}'.format(sensor_channel, box_list[0]))
```
I get the result like this:
```
CAM_FRONT channel path = J:\data\lyft\nuscenes-devkit\notebooks\..\..\perception\one_scene\train_images\host-a101_cam0_1241893239450000006.jpeg
CAM_FRONT channel box = label: nan, score: nan, xyz: [-17.98, 1.52, 27.38], wlh: [1.70, 3.89, 1.49], rot axis: [0.01, -0.70, 0.72], ang(degrees): 178.16, ang(rad): 3.11, vel: nan, nan, nan, name: car, token: 9f1434fa2688e89f7770958b13a24cb6b1d815a31ab1efb6346565799482d350
LIDAR_TOP channel path = J:\data\lyft\nuscenes-devkit\notebooks\..\..\perception\one_scene\train_lidar\host-a101_lidar1_1241893239502712366.bin
LIDAR_TOP channel box = label: nan, score: nan, xyz: [27.67, -16.85, -0.47], wlh: [1.55, 4.05, 1.41], rot axis: [-0.02, 0.03, 1.00], ang(degrees): -39.19, ang(rad): -0.68, vel: nan, nan, nan, name: car, token: fdfa2d9494bd4b2bc183522bc8a71ee02e0eeb451dfb5f91d057ffcdb8a8e7fc
CAM_BACK channel path = J:\data\lyft\nuscenes-devkit\notebooks\..\..\perception\one_scene\train_images\host-a101_cam3_1241893239500000006.jpeg
CAM_BACK channel box = label: nan, score: nan, xyz: [17.29, 0.37, 27.02], wlh: [1.55, 4.05, 1.41], rot axis: [0.84, -0.39, 0.38], ang(degrees): 101.59, ang(rad): 1.77, vel: nan, nan, nan, name: car, token: fdfa2d9494bd4b2bc183522bc8a71ee02e0eeb451dfb5f91d057ffcdb8a8e7fc
```
I feel confused about the xyz of the box_list[0], so I look into the source code.
```
box_list = []
for box in boxes:
if flat_vehicle_coordinates:
# Move box to ego vehicle coord system parallel to world z plane
ypr = Quaternion(pose_record["rotation"]).yaw_pitch_roll
yaw = ypr[0]

box.translate(-np.array(pose_record["translation"]))
box.rotate_around_origin(Quaternion(scalar=np.cos(yaw / 2), vector=[0, 0, np.sin(yaw / 2)]).inverse)

else:
# Move box to ego vehicle coord system
box.translate(-np.array(pose_record["translation"]))
box.rotate_around_origin(Quaternion(pose_record["rotation"]).inverse)

# Move box to sensor coord system
box.translate(-np.array(cs_record["translation"]))
box.rotate_around_origin(Quaternion(cs_record["rotation"]).inverse)

if sensor_record["modality"] == "camera" and not box_in_image(
box, cam_intrinsic, image_size, vis_level=box_vis_level
):
continue

box_list.append(box)
```
The parameter flat_vehicle_coordinates is False, so I think the final box_list‘s xyz is in sensor coord system.
But the result I find the z is 27.38 or 27.02 (m ?)
I want to know where to find the definition of each sensor coord system ? I'm confused about the result, especially the z axis.

Thanks for your contribution for this big dataset very much !

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the get_sample_data call and the shown coordinate-transform block, then compare the roles of pose_record, cs_record, sensor_record, and cam_intrinsic. Done means the coordinate-system definitions and the meaning and units of returned box xyz values, especially z, are explicitly documented for each sensor channel.

Written by the indexing model from the issue text.

Assessment

Tech stack
jupyter-notebook, python
Domain
autonomous-driving, computer-vision, documentation
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.