pytorch / pytorch/pytorch

Feature request: built-in loss extrapolator

Open
#188,216 0 comments 0 reactions 0 assignees View on GitHub
bot-triaged feature module: loss needs design triaged
Dominant language
Python
Stars
103k
Forks
29.5k
PR merge metrics
PR metrics pending

Description

### 🚀 The feature, motivation and pitch

According to the Neural Scaling Laws paper (Kaplan, et. al. 2020) we can make a pretty good guess at what a model's loss will be after just a few training steps. It would be nice to include this in the training printout as like a "estimated final loss" or something.

### Alternatives

_No response_

### Additional context

_No response_

Contributor guide

Open the contributing guide

Research direction

The issue names only the training printout and does not identify a file, test, estimator interface, or output format. Start by locating the code that produces training loss output and review how training metrics are reported. Before implementation, clarify the scaling-law method, required inputs, display format, and tests that would define the estimate as done.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.