pytorch / pytorch/kineto

Retrieving GPU power counters via `CUpti_ActivityEnvironment`

Open
#997 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

OSS contribution requested
Dominant language
C++
Stars
992
Forks
270
Avg merge
2d 11h
Merged PRs (30d)
12

Description

Recently there is some interest around GPU power draw (e.g., https://github.com/pytorch/pytorch/pull/132936), because large scale training consumes power at an unprecedented scale that affects datacenter power delivery and the grid.

Along that line, I was wondering if Kineto is interested in incorporating power draw information via the CUPTI activity API, specifically CUpti_ActivityEnvironment.

I'm no expert in CUPTI or Kineto, but since CUPTI activities are already being used in the codebase, I thought it might not be too hard to add this. If this aligns with the project's direction, I am willing to contribute this feature myself. But to be real honest, I have practically zero experience with CUPTI or using Kineto, so I might need a little handholding and time.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by examining Kineto's existing CUPTI activity integration and the CUPTI_ActivityEnvironment API documentation. Determine how power-counter records would be collected and represented, then define validation for exposing GPU power-draw data in profiler output; the issue names no files or tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
observability, performance
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.