Add support for 0-SM pipeline parallelism using copy-engine communication

The current pipeline-parallel communication path appears to rely on SMs for transfer/coordination work. For some workloads, that leaves compute resources on the table and reduces effective occupancy.

I’d like to propose support for 0-SM pipeline parallelism, where pipeline-parallel communication is moved onto the copy engine instead of consuming SM time. The goal is to free up more SMs for compute while still preserving correct PP synchronization and overlap.

Motivation:

• Reduce SM pressure from pipeline-parallel communication
• Increase effective compute capacity for the forward/backward path
• Improve throughput for models where PP communication is a meaningful bottleneck

What I’m asking for:

• Add a mode or backend path that uses the copy engine for PP communication
• Keep a fallback path for devices or configurations that cannot use it
• Ensure the implementation preserves correctness and ordering guarantees

Useful follow-ups:

• Benchmark SM utilization and end-to-end throughput against the current PP path
• Document the supported GPU architectures and any limitations
• Validate that the new path works with common PP schedules and model sizes

If useful, I can also help refine this into a more implementation-focused proposal or a benchmark-oriented issue.

No implementation files, tests, or entry points are named. Start by locating the current pipeline-parallel communication path and its device capability checks; compare a copy-engine path with the existing path across common PP schedules, then verify synchronization, fallback behavior, SM utilization, throughput, and documented GPU limitations.