Evaluation results inconsistent with reported metrics in the paper

Hi, thanks for the great work on CADFusion and for releasing the code!

I’m trying to reproduce the evaluation results reported in the paper using the evaluation scripts provided in this repo. However, I noticed that the metrics I obtained are quite different from the values in the paper.

My evaluation setup

• Using the official evaluation script:
  src/test/dist_eval.py

• Model: (e.g., CADFusion v1.0 checkpoint from the repo huggingface)

• Dataset: (strictly following the readme instructions)

• Command I ran:

  python3 src/test/dist_eval.py \
    --fake ${FAKE_VISUAL_DIR} \
    --real ${REAL_GT_DIR} \
    --output ${EVAL_OUTPUT} \
    --batch_size 128 \
    > "${LOG_DIR}/evaluation.log" 2>&1
  

Results I obtained

The metrics I got are:

{
    'avg-CD': 0.3818102777004242,
    'COV-CD': 0.17968350648880005,
    'MMD-CD': 0.0164920873939991,
    'JSD': 0.018295722019258776
}

Difference from paper results

These are noticeably different from the results reported in the CADFusion paper (especially for CD-based metrics).

Question

Could you please help clarify:

1. Are there additional preprocessing steps, normalization, or sampling procedures required to reproduce the paper results?
2. Are the default evaluation settings in the repo identical to those used in the paper?
3. Is there any known issue or update regarding the evaluation script or metrics?

Any guidance would be greatly appreciated — I’d like to ensure that I’m running the evaluation correctly.

Thanks again for the great work!

Start with src/test/dist_eval.py and the README instructions, then run the reported command with the CADFusion v1.0 checkpoint and the stated dataset paths. Compare preprocessing, normalization, sampling, and default evaluation settings with those used for the paper. Done means the discrepancy is explained and the repository instructions or evaluation behavior clearly identifies how to reproduce the reported metrics.