3DTopia / 3DTopia/DynamicCity

Request for Metric^2D / Metric^3D evaluation code (IS/FID/KID/P/R)

Open
#20 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
249
Forks
12
PR merge metrics
No merged PRs in 30d

Description

Hi, thanks for releasing DynamicCity.

I checked the released repository, but could not find the evaluation code for the paper's Metric^2D and Metric^3D results. The current code seems to include VAE training metrics such as mIoU/binIoU, but I could not find scripts/configs for the paper's feature-based generation metrics:

- Metric^2D: rendering 3D/4D occupancy into 2D semantic images, then extracting features with 2D networks such as InceptionV3 / VGG-16.
- Metric^3D: extracting features directly from 3D occupancy with a 3D network such as MinkowskiUNet.
- Computing IS, FID, KID, Precision, and Recall in those 2D/3D feature spaces, including the exact pretrained checkpoints/features used.

Could you please release the evaluation scripts, pretrained feature extractors, rendering settings, and command examples needed to reproduce the 2D and 3D metrics reported in the paper?

I saw earlier general requests about FID/metrics, but this request is specifically about the Metric^2D / Metric^3D protocols and feature extractors used in the paper.

This would be very helpful for comparing follow-up 4D occupancy generation methods under the same protocol. Thanks!

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by locating the existing VAE evaluation metrics, including mIoU/binIoU, and compare them with the paper's Metric^2D and Metric^3D protocols. The work is complete when evaluation scripts, feature-extractor checkpoints, rendering settings, and command examples reproduce IS, FID, KID, Precision, and Recall in both feature spaces.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
computer-vision, machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.