vllm-project / vllm-project/afd-plugin
[Feature]: Enable multi-stream parallelism for CAMP2P A2E/E2A on FFN side
@yujuancao07 is already working on this.
Since Jul 22, 2026.
- Dominant language
- Python
- Stars
- 228
- Forks
- 48
- Avg merge
- 22h 23m
- Merged PRs (30d)
- 44
Description
Feature, motivation, and pitch
Support enabling multi-stream parallelism for CAMP2P A2E/E2A communication on the FFN side.
When the FFN side is the compute bottleneck, A2E and E2A can overlap with FFN-side computation through multi-stream execution. This should reduce per-layer execution time by improving communication/computation overlap in CAMP2P flows.
Proposed change
Add support for FFN-side CAMP2P A2E/E2A multi-stream parallel execution.
Expected behavior:
- Allow A2E and E2A on the FFN side to run on separate streams where applicable.
- Overlap A2E/E2A with FFN computation when FFN is the bottleneck.
- Keep the feature configurable so it can be enabled only for supported CAMP2P scenarios.
- Ensure stream synchronization is explicit and safe around tensor ownership, data dependencies, and layer boundaries.
Potential implementation areas may include the FFN model runner/worker path and CAMP2P communication helpers, depending on the current plugin-owned abstraction.
vLLM compatibility and extension points
Preferred extension point: plugin-owned FFN runner/worker and CAMP2P communication logic.
Compat shim needed: only if the current plugin interface does not expose enough stream or scheduling control.
Monkey patch needed: avoid modifying the vLLM v0.19.1 source tree. If a patch is unavoidable, keep it isolated, version-guarded, idempotent, and documented.
Alternatives considered
- Keep A2E/E2A serialized with FFN execution. This is simpler but leaves performance on the table when FFN is compute-bound.
- Apply global communication stream changes. This may be broader than necessary and could affect non-CAMP2P paths, so the first implementation should prefer a scoped FFN-side CAMP2P change.
Validation plan
- Add or update CPU-safe unit tests for configuration gating and scheduling/stream selection logic where possible.
- Add GPU/NPU-gated validation for CAMP2P FFN-side execution with multi-stream enabled.
- Compare per-layer execution time with and without FFN-side A2E/E2A multi-stream parallelism in an FFN-compute-bottleneck scenario.
- Verify correctness against the existing serialized path, including output consistency and synchronization safety.
Before submitting
- I searched existing issues and RFCs.
- I identified whether this belongs in plugin-owned code, compat helpers, or compat patches.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.