MCG-NJU / MCG-NJU/SparseOcc

报告一些关于 RayIoU 的小错误,希望指标计算细节社区能统一一下(已⭐️)

Open
#71 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
443
Forks
37
PR merge metrics
No merged PRs in 30d

Description

  1. EgoPoseDataset 生成的激光雷达原点可能在场景外,作者意图应该还是想限制在场景内的吧,但是代码里只考虑俯视平面,没考虑到自车的姿态,比如在下坡场景中远处的位置实测是有部分原点在地下或上天的,在我的实验中这里会带来 (0.426182-0.424588) 的 1e-3 级的差别
# if np.abs(origin_tf[0]) < 39 and np.abs(origin_tf[1]) < 39:
if np.abs(origin_tf[0]) < 39 and np.abs(origin_tf[1]) < 39 and (origin_tf[2] > -1 and origin_tf[2] < 5.4):
    output_origin_list.append(origin_tf)
  1. calc_rayiou 中要不要也像计算普通 iou 一样显式置 nan,这只对计算迷你验证集上计算指标有影响,完整验证集上不会出现某个真值类从不出现的情况
def per_class_iou(hist):
    result = np.diag(hist) / (hist.sum(1) + hist.sum(0) - np.diag(hist))
# ----------------------------------------------------------------------------------------------------
    result[hist.sum(1) == 0] = float('nan')
# ----------------------------------------------------------------------------------------------------
    return result

def calc_rayiou(pcd_pred_list, pcd_gt_list, occ_class_names):
....
    iou_list = []
    for j, threshold in enumerate(thresholds):
# ----------------------------------------------------------------------------------------------------
        # iou_list.append((tp_cnt[j] / (gt_cnt + pred_cnt - tp_cnt[j]))[:-1])
        iou = (tp_cnt[j] / (gt_cnt + pred_cnt - tp_cnt[j])
        iou[gt_cnt == 0] = float('nan')
        iou_list.append(iou[:-1])
# ----------------------------------------------------------------------------------------------------
    return iou_list

  1. 虽然听着很扯,但是下面这段冗余的代码似乎会导致奇怪的 Bug(当前计算指标的代码里用 torch.cuda.empty_cache() 可以解决此 Bug),删除下面的冗余代码后可以不用 empty_cache
Image

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by locating the EgoPoseDataset origin-generation code and the calc_rayiou and per_class_iou entry points mentioned in the issue. Reproduce the mini-validation behavior, then investigate the origin bounds, NaN handling for absent ground-truth classes, and the redundant code shown in the report. Done means the reported metric differences and cache-related bug are resolved without requiring torch.cuda.empty_cache().

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
computer-vision
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.