Clarify train loss vs reported Rel-L2 (NASA-CRM / AhmedML / DrivAerML)
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 34
- Forks
- 6
- PR merge metrics
- No merged PRs in 30d
Description
Hi — thanks for the paper and code. We're reimplementing the industrial benchmarks and want to match your training/eval protocol.
Context. Table 9 lists training Loss = Relative L2, Appendix A.3 says targets are standardized (zero mean / unit variance), and Table 4 reports per-field Rel-L2 (%). It's unclear how these interact, and the release doesn't fully cover NASA-CRM / field-wise metrics for every case.
For each of NASA-CRM, AhmedML, and DrivAerML, could you clarify:
Training loss
- Is Rel-L2 applied in normalized space, physical space (after denorm), or is training actually MSE on normalized targets with Rel-L2 only used at eval?
- Is the objective a single joint Rel-L2 over all output channels, or an average of per-field Rel-L2 (e.g.
0.5*(RelL2(cp)+RelL2(cf)))? - For vector fields, does the model predict components or magnitude?
- NASA-CRM:
[cp, cx, cy, cz]vs[cp, |cf|]with|cf|=√(cx²+cy²+cz²)? - AhmedML / DrivAerML surface: pressure
ps+ wall shearτas a 3-vector or|τ|? - AhmedML / DrivAerML volume: velocity
uas a 3-vector or|u|, plus volume pressurepv?
- NASA-CRM:
- Any special field/channel weighting?
Reported metrics (Table 4 / App. A.3)
- Always computed in physical units after inverse standardization?
- For vector quantities (
Cf,τ,u): Rel-L2 on the full vector, or on the per-node magnitude first? - Per-field Rel-L2 = mean over test samples of
‖ŷ−y‖₂ / ‖y‖₂on that field only? - With amortized subset training (Ahmed/DrivAer), are metrics still on the full mesh?
A short answer per dataset (or a pointer to the loss/metric code) would be perfect. Thanks!
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating the loss and metric implementations used for NASA-CRM, AhmedML, and DrivAerML, then compare them with the standardization and benchmark descriptions in Appendix A.3 and Table 4. Done means documenting the training-space, channel aggregation, vector handling, weighting, and full-mesh evaluation choices for each dataset, or linking to the relevant code.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Documentation
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100