microsoft / microsoft/vidur

Regarding the calculation of the duration in the profiling phase

Open
#53 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
682
Forks
128
Avg merge
9h 44m
Merged PRs (30d)
1

Description

Hello Vidur,

Thank you for sharing your work. While reading the code, I encountered a question.

I am analyzing the profiling part of the code. The profiling is divided into two parts: MLP and attention.
When calculating the duration for MLP, it first finds all the children events, then finds the correlation for each child, and sums the duration of all correlated events to get the total duration for the event. (The function get_operation_time_stats() in Vidur)
However, for attention, it uses the sarathi module, where the implementation sums up all the CUDA runtime durations using sum([e.cuda_time_total for e in trace.key_averages()]). This approach sums all CUDA runtime durations. (The function handle_trace() in Sarathi)
However, for MLP, it does not sum all CUDA runtime durations.
Is my understanding correct?
These two methods seem inconsistent. How were these approaches considered?

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading get_operation_time_stats() in Vidur and handle_trace() in Sarathi, then compare how each profiling path derives duration from correlated events and CUDA runtime entries. Confirm whether the difference is intentional; done means resolving the inconsistency or documenting the agreed behavior with coverage for the relevant calculation.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.