aws / aws/amazon-sagemaker-examples

[Bug Report] Wrong function call in model_parallel_v2

Open
#4,791 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
11k
Forks
7k
Avg merge
8h 29m
Merged PRs (30d)
8

Description

Hi all,
There is a bug in the following py file: training/distributed_training/pytorch/model_parallel_v2/shared-scripts/logging_utils.py
Line 151 states the following: `avg_tflops = compute_tflops(avg_throughput, num_params, world_size, batch_seqlen)`
But the function definition in: training/distributed_training/pytorch/model_parallel_v2/shared-scripts/train_utils.py
at line 36 is the following: `def compute_tflops(args, global_batch_size, step_time, world_size):`

The arguments of the function call should be adapted, at least args shall be passed as the first one (or a new function as to be defined).

Contributor guide

Open the contributing guide

Research direction

Read training/distributed_training/pytorch/model_parallel_v2/shared-scripts/logging_utils.py around line 151 and train_utils.py around line 36, then trace how compute_tflops is intended to receive its inputs. Align the call and definition without changing the reported metric semantics, and verify that the model_parallel_v2 training example runs successfully.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
distributed-systems, machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.