vllm-project / vllm-project/afd-plugin

[Feature]: Enable multi-stream parallelism for CAMP2P A2E/E2A on FFN side

Open
#68 0 comments 0 reactions 1 assignee View on GitHub

@yujuancao07 is already working on this.

Since Jul 22, 2026.

enhancement
Dominant language
Python
Stars
228
Forks
48
Avg merge
22h 23m
Merged PRs (30d)
44

Description

Feature, motivation, and pitch

Support enabling multi-stream parallelism for CAMP2P A2E/E2A communication on the FFN side.

When the FFN side is the compute bottleneck, A2E and E2A can overlap with FFN-side computation through multi-stream execution. This should reduce per-layer execution time by improving communication/computation overlap in CAMP2P flows.

Proposed change

Add support for FFN-side CAMP2P A2E/E2A multi-stream parallel execution.

Expected behavior:

  • Allow A2E and E2A on the FFN side to run on separate streams where applicable.
  • Overlap A2E/E2A with FFN computation when FFN is the bottleneck.
  • Keep the feature configurable so it can be enabled only for supported CAMP2P scenarios.
  • Ensure stream synchronization is explicit and safe around tensor ownership, data dependencies, and layer boundaries.

Potential implementation areas may include the FFN model runner/worker path and CAMP2P communication helpers, depending on the current plugin-owned abstraction.

vLLM compatibility and extension points

Preferred extension point: plugin-owned FFN runner/worker and CAMP2P communication logic.

Compat shim needed: only if the current plugin interface does not expose enough stream or scheduling control.

Monkey patch needed: avoid modifying the vLLM v0.19.1 source tree. If a patch is unavoidable, keep it isolated, version-guarded, idempotent, and documented.

Alternatives considered

  • Keep A2E/E2A serialized with FFN execution. This is simpler but leaves performance on the table when FFN is compute-bound.
  • Apply global communication stream changes. This may be broader than necessary and could affect non-CAMP2P paths, so the first implementation should prefer a scoped FFN-side CAMP2P change.

Validation plan

  • Add or update CPU-safe unit tests for configuration gating and scheduling/stream selection logic where possible.
  • Add GPU/NPU-gated validation for CAMP2P FFN-side execution with multi-stream enabled.
  • Compare per-layer execution time with and without FFN-side A2E/E2A multi-stream parallelism in an FFN-compute-bottleneck scenario.
  • Verify correctness against the existing serialized path, including output consistency and synchronization safety.

Before submitting

  • I searched existing issues and RFCs.
  • I identified whether this belongs in plugin-owned code, compat helpers, or compat patches.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.