Exclude data staging from runtime in hpc_3.0.0/common.yaml checks
Open
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 43
- Forks
- 60
- Avg merge
- 19h 56m
- Merged PRs (30d)
- 3
Description
hpc_3.0.0/common.yaml doesn't exist yet,
but I want to remind in this issue the data staging has been excluded from measuring runtime in HPC, so this part:
hpc_2.0.0/common.yaml#L77-L87
should be updated accordingly.
The new expected order in the logs is:
- init_start
- staging_start
- staging_stop
- init_stop
- run_start
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading mlperf_logging/compliance_checker/hpc_2.0.0/common.yaml lines 77-87 and compare the stated expected log order. Create the corresponding hpc_3.0.0/common.yaml checks so staging_start and staging_stop occur between init_start and init_stop without counting staging as runtime. Done means the checks accept the listed order.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- testing-qa
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100