mlcommons / mlcommons/logging

Document RCPs method of production

Open
#116 6 comments 0 reactions 1 assignee View on GitHub

@nv-rborkar is already working on this.

Since Oct 19, 2021.

v1.1
Dominant language
Python
Stars
43
Forks
60
Avg merge
19h 56m
Merged PRs (30d)
3

Description

It would be useful if each RCP point also notes the method of production that was used to generate it, in the RCP jsons.
Intention would be to have a path of reproducibility if needed or record for future reference- especially when at submission time we will have few PRs for different submitters with new RCPs, probably run on different HW platforms & different framework versions.

Method of production can be noted via additional fields per RCP point such as:

  • "hw_notes": for example tpuv3-128 or DGXA100 etc.
  • "framework": for example TF2.1.1, pytorch etc
  • "reference git commit hash": To note the reference commit that was used to generate the RCP
  • "notes": For optional details, for example if a rcp is from a submission code repo vs the reference repo for temporary reasons

This would avoid us all scratching our heads post v1.0 about different RCP points.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.