isl-org / isl-org/Open3D-ML

`run_pipeline.py` hangs on Open3D's `iou_bev` invocation in PyTorch

Open
#340 5 comments 0 reactions 1 assignee Claimed by @sanskar107 View on GitHub
Dominant language
Python
Stars
2.3k
Forks
365
Avg merge
4h 6m
Merged PRs (30d)
1

Description

The [`iou_bev` invocation inside `datasets/utils/operations.py`'s `box_collision_test()`](https://github.com/isl-org/Open3D-ML/blob/200bee315a55772a9b40f1304d9165242f896786/ml3d/datasets/utils/operations.py#L426) hangs on invocation with no CPU or memory usage, blocking all forward progress.

This can be reproduced using [the following `Dockerfile`](https://raw.githubusercontent.com/kylevedder/Open3D-ML/05f16366164486ec7d8c06c5ae7eff54bba97fa4/Dockerfile), which codifies the instructions provided in the README, and then the following command run inside the container with this repo mounted at `/Open3D-ML`:

```
python scripts/run_pipeline.py torch -c ml3d/configs/pointpillars_kitti.yml --dataset_path /data/kitti_second/kitti/ --pipeline ObjectDetection
```

which produces

```
Using external Open3D-ML in /Open3D-ML
regular arguments
batch_size: null
cfg_dataset: null
cfg_file: ml3d/configs/pointpillars_kitti.yml
cfg_model: null
cfg_pipeline: null
ckpt_path: null
dataset: null
dataset_path: /data/kitti_second/kitti/
device: gpu
framework: torch
main_log_dir: null
max_epochs: null
mode: null
model: null
pipeline: ObjectDetection
split: train

extra arguments
{}

INFO - 2021-08-05 04:50:41,756 - object_detection - DEVICE : cuda
INFO - 2021-08-05 04:50:41,756 - object_detection - Logging in file : ./logs/PointPillars_KITTI_torch/log_train_2021-08-05_04:50:41.txt
INFO - 2021-08-05 04:50:41,756 - kitti - Found 3712 pointclouds for training
INFO - 2021-08-05 04:50:41,757 - object_detection - Initializing from scratch.
INFO - 2021-08-05 04:50:41,758 - object_detection - Writing summary in train_log/00068_PointPillars_KITTI_torch.
INFO - 2021-08-05 04:50:41,759 - object_detection - Started training
INFO - 2021-08-05 04:50:41,759 - object_detection - === EPOCH 0/200 ===
INFO - 2021-08-05 04:50:41,759 - object_detection - after model.train()
training: 0%| | 0/834 [00:00

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.