mlcommons / mlcommons/logging

Correctly implement lower convex envelope in RCP pruning logic

Open Beginner friendly
#368 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
43
Forks
60
Avg merge
19h 56m
Merged PRs (30d)
3

Description

As the comment at https://github.com/mlcommons/logging/blob/master/mlperf_logging/rcp_checker/rcp_checker.py#L246 says, the loop does not correctly implement the "lower convex envelope" that was specified in the original RCP specification. There is an off by one error. If a point X needs to be pruned because it is greater than the interpolation of X-1 and X+1, then the point to the left of point X (X-1) needs to be retested against the interpolation of points X-2 and X+1. The increment at line 256 should be in an else clause.
This bug leads to bad RCPs not getting pruned which leads to submissions getting either unfairly rejected or unfairly "scaled" when they have a global batch size near the bad RCP point

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in mlperf_logging/rcp_checker/rcp_checker.py around lines 246-256 and read the nearby comment alongside the original RCP specification. Trace the lower convex envelope loop and verify that the preceding point is retested after a point is pruned. Done means bad RCP points are pruned correctly, including cases near the global batch-size boundary, with the relevant checker tests passing.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
65/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.