Tracker for Debuggability Improvement

🚀 Key Features

This is a tracker issue for the improvement for debuggability.

• #299

graph TD;
DP1-->DP2;
DP2-->DP3;
DP2-->DP4;
DP3-->DP5;
DP4-->DP6;
DP5-->DP6;
DP6-->output;

can be printed out as the following

>>> print_graph(traverse(dp6))
DP1 -> DP2 -> DP3 -> DP5
         \             \
         DP4 --------> DP6 ->

• Find a way to differentiate DataPipe instance in the graph
  • Provide shorter name for each DataPipe instance (name is now modified within exception)
  • Print input arguments of the DataPipe within exceptions
    https://github.com/pytorch/pytorch/pull/75618
  • Provide unique name using incremental number for the instance of the same DataPipe class
• Improve back tracing
  • During Iteration, remove this profiling part within back tracing https://github.com/pytorch/pytorch/blob/11231b0f935c092598c994a4bab686485aac1856/torch/utils/data/_typing.py#L340
  • Reduce back tracing to highlight which DataPipe is the culprit when __len__ is invoked.
• #210

Nice to Haves

This section is tracking potential features that we may want

• Handling mixed usage of IterDataPipe, MapDataPipe, torcharrow DataFrame
  • Are users able to clearly differentiate these when they are using a mixture of these classes?
• Connect profiling result with graph
  • Different colors for nodes based on their performance (similar to TensorBoard)

Motivation, pitch

This would help our users and developers to easily understand what's going on with the pipeline.
Feel free to post more request for debuggability.

Alternatives

No response

Additional context

No response

cc: @NivekT @VitalyFedyunin

This is a broad tracker rather than a single implementation task; review the unchecked back-tracing items and the existing print_graph(traverse(dp6)) example first. The only named code location is torch/utils/data/_typing.py#L340, with related work in #210 and pull request #75618. Done criteria are not defined because the remaining ideas require separate scope and decisions.