thuml / thuml/Transolver-3

Clarify train loss vs reported Rel-L2 (NASA-CRM / AhmedML / DrivAerML)

Open
#4 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
34
Forks
6
PR merge metrics
No merged PRs in 30d

Description

Hi — thanks for the paper and code. We're reimplementing the industrial benchmarks and want to match your training/eval protocol.

Context. Table 9 lists training Loss = Relative L2, Appendix A.3 says targets are standardized (zero mean / unit variance), and Table 4 reports per-field Rel-L2 (%). It's unclear how these interact, and the release doesn't fully cover NASA-CRM / field-wise metrics for every case.

For each of NASA-CRM, AhmedML, and DrivAerML, could you clarify:

Training loss
  1. Is Rel-L2 applied in normalized space, physical space (after denorm), or is training actually MSE on normalized targets with Rel-L2 only used at eval?
  2. Is the objective a single joint Rel-L2 over all output channels, or an average of per-field Rel-L2 (e.g. 0.5*(RelL2(cp)+RelL2(cf)))?
  3. For vector fields, does the model predict components or magnitude?
    • NASA-CRM: [cp, cx, cy, cz] vs [cp, |cf|] with |cf|=√(cx²+cy²+cz²)?
    • AhmedML / DrivAerML surface: pressure ps + wall shear τ as a 3-vector or |τ|?
    • AhmedML / DrivAerML volume: velocity u as a 3-vector or |u|, plus volume pressure pv?
  4. Any special field/channel weighting?
Reported metrics (Table 4 / App. A.3)
  1. Always computed in physical units after inverse standardization?
  2. For vector quantities (Cf, τ, u): Rel-L2 on the full vector, or on the per-node magnitude first?
  3. Per-field Rel-L2 = mean over test samples of ‖ŷ−y‖₂ / ‖y‖₂ on that field only?
  4. With amortized subset training (Ahmed/DrivAer), are metrics still on the full mesh?

A short answer per dataset (or a pointer to the loss/metric code) would be perfect. Thanks!

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by locating the loss and metric implementations used for NASA-CRM, AhmedML, and DrivAerML, then compare them with the standardization and benchmark descriptions in Appendix A.3 and Table 4. Done means documenting the training-space, channel aggregation, vector handling, weighting, and full-mesh evaluation choices for each dataset, or linking to the relevant code.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Documentation
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.