报告一些关于 RayIoU 的小错误,希望指标计算细节社区能统一一下(已⭐️)
Open
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 443
- Forks
- 37
- PR merge metrics
- No merged PRs in 30d
Description
- EgoPoseDataset 生成的激光雷达原点可能在场景外,作者意图应该还是想限制在场景内的吧,但是代码里只考虑俯视平面,没考虑到自车的姿态,比如在下坡场景中远处的位置实测是有部分原点在地下或上天的,在我的实验中这里会带来 (0.426182-0.424588) 的 1e-3 级的差别
# if np.abs(origin_tf[0]) < 39 and np.abs(origin_tf[1]) < 39:
if np.abs(origin_tf[0]) < 39 and np.abs(origin_tf[1]) < 39 and (origin_tf[2] > -1 and origin_tf[2] < 5.4):
output_origin_list.append(origin_tf)
- calc_rayiou 中要不要也像计算普通 iou 一样显式置 nan,这只对计算迷你验证集上计算指标有影响,完整验证集上不会出现某个真值类从不出现的情况
def per_class_iou(hist):
result = np.diag(hist) / (hist.sum(1) + hist.sum(0) - np.diag(hist))
# ----------------------------------------------------------------------------------------------------
result[hist.sum(1) == 0] = float('nan')
# ----------------------------------------------------------------------------------------------------
return result
def calc_rayiou(pcd_pred_list, pcd_gt_list, occ_class_names):
....
iou_list = []
for j, threshold in enumerate(thresholds):
# ----------------------------------------------------------------------------------------------------
# iou_list.append((tp_cnt[j] / (gt_cnt + pred_cnt - tp_cnt[j]))[:-1])
iou = (tp_cnt[j] / (gt_cnt + pred_cnt - tp_cnt[j])
iou[gt_cnt == 0] = float('nan')
iou_list.append(iou[:-1])
# ----------------------------------------------------------------------------------------------------
return iou_list
- 虽然听着很扯,但是下面这段冗余的代码似乎会导致奇怪的 Bug(当前计算指标的代码里用 torch.cuda.empty_cache() 可以解决此 Bug),删除下面的冗余代码后可以不用 empty_cache
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating the EgoPoseDataset origin-generation code and the calc_rayiou and per_class_iou entry points mentioned in the issue. Reproduce the mini-validation behavior, then investigate the origin bounds, NaN handling for absent ground-truth classes, and the redundant code shown in the report. Done means the reported metric differences and cache-related bug are resolved without requiring torch.cuda.empty_cache().
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- computer-vision
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100