Regarding the calculation of the duration in the profiling phase
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 682
- Forks
- 128
- Avg merge
- 9h 44m
- Merged PRs (30d)
- 1
Description
Hello Vidur,
Thank you for sharing your work. While reading the code, I encountered a question.
I am analyzing the profiling part of the code. The profiling is divided into two parts: MLP and attention.
When calculating the duration for MLP, it first finds all the children events, then finds the correlation for each child, and sums the duration of all correlated events to get the total duration for the event. (The function get_operation_time_stats() in Vidur)
However, for attention, it uses the sarathi module, where the implementation sums up all the CUDA runtime durations using sum([e.cuda_time_total for e in trace.key_averages()]). This approach sums all CUDA runtime durations. (The function handle_trace() in Sarathi)
However, for MLP, it does not sum all CUDA runtime durations.
Is my understanding correct?
These two methods seem inconsistent. How were these approaches considered?
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading get_operation_time_stats() in Vidur and handle_trace() in Sarathi, then compare how each profiling path derives duration from correlated events and CUDA runtime entries. Confirm whether the difference is intentional; done means resolving the inconsistency or documenting the agreed behavior with coverage for the relevant calculation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100