Request for Metric^2D / Metric^3D evaluation code (IS/FID/KID/P/R)
- Dominant language
- Python
- Stars
- 249
- Forks
- 12
- PR merge metrics
- No merged PRs in 30d
Description
Hi, thanks for releasing DynamicCity.
I checked the released repository, but could not find the evaluation code for the paper's Metric^2D and Metric^3D results. The current code seems to include VAE training metrics such as mIoU/binIoU, but I could not find scripts/configs for the paper's feature-based generation metrics:
- Metric^2D: rendering 3D/4D occupancy into 2D semantic images, then extracting features with 2D networks such as InceptionV3 / VGG-16.
- Metric^3D: extracting features directly from 3D occupancy with a 3D network such as MinkowskiUNet.
- Computing IS, FID, KID, Precision, and Recall in those 2D/3D feature spaces, including the exact pretrained checkpoints/features used.
Could you please release the evaluation scripts, pretrained feature extractors, rendering settings, and command examples needed to reproduce the 2D and 3D metrics reported in the paper?
I saw earlier general requests about FID/metrics, but this request is specifically about the Metric^2D / Metric^3D protocols and feature extractors used in the paper.
This would be very helpful for comparing follow-up 4D occupancy generation methods under the same protocol. Thanks!
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by locating the existing VAE evaluation metrics, including mIoU/binIoU, and compare them with the paper's Metric^2D and Metric^3D protocols. The work is complete when evaluation scripts, feature-extractor checkpoints, rendering settings, and command examples reproduce IS, FID, KID, Precision, and Recall in both feature spaces.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- computer-vision, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100