[Feature]: Add CAMAsync multi-stream execution on a shared NPU stream pipeline

Feature, motivation, and pitch

Add multi-stream execution for CAMAsyncAFDConnector, with a design-first requirement to determine how much infrastructure can be shared with the CAMP2P FFN-side multi-stream work proposed in #68.

CAMAsync currently has pipeline-level asynchronous behavior, but its Attention and connector-driven FFN paths do not expose a reviewed, reusable stream/event scheduler. On the FFN side, async_dispatch_recv, expert computation, and async_combine_send are orchestrated serially by the runner loop and followed by a device-wide synchronization. On the Attention side, dispatch/combine can be delayed across layers or microbatch stages, but stream placement, dependencies, buffer lifetime, and synchronization are not represented by a common runtime abstraction.

The goal is to overlap CAM communication with model computation safely, while avoiding a second connector-specific multi-stream implementation. The preferred outcome is one plugin-owned NPU stream-pipeline framework used by both CAMP2P (#68) and CAMAsync, with only operator invocation and transfer-state handling remaining connector-specific.

Related issue: #68.

Design-first requirement

Implementation must not begin with connector-local stream creation. Phase 0 must produce and review a short design that answers:

1. What is common between CAMP2P and CAMAsync?
   • stream creation and ownership;
   • receive/compute/send phase scheduling;
   • event recording and waiting;
   • tensor and transfer-state lifetime tracking;
   • bounded in-flight work;
   • exception cleanup, shutdown, and single-stream fallback;
   • profiling and correctness instrumentation.
2. What must remain connector-specific?
   • CAMP2P a2e/e2a arguments, HCCL group selection, and CAMP2PTransferState;
   • CAMAsync async_dispatch_recv/async_combine_send and async_dispatch_send/async_combine_recv arguments, pending FIFO state, routing metadata, dynamic-quant buffers, and external CAM operator constraints.
3. Do the external umdk_cam_op_lib async operators honor the active torch.npu stream, and are their communicator/handle and buffers safe for concurrent in-flight operations?
4. Which dependencies require NPU events, which require operator-provided completion, and where—if anywhere—is host synchronization unavoidable?
5. Which combinations are supported initially: Attention side, FFN side, async MoE ubatching on/off, dynamic quantization, DP/PCP/TP/SP, and eager-only execution?

The design must include execution timelines/DAGs for both #68 CAMP2P and CAMAsync, plus a decision table showing shared versus connector-specific components. If CAM operators cannot safely execute on caller-selected streams, document the limitation and retain the common scheduling/lifecycle shell with a narrow CAM-specific synchronization adapter rather than duplicating the whole framework.

Proposed shared architecture

Introduce a plugin-owned NPU stream-pipeline layer, with final names chosen during design review. The intended ownership is:

AFDNPUStreamPipelineExecutor
  owns:
    compute / receive-communication / send-communication streams
    reusable NPU event pool
    bounded in-flight work slots
    dependency scheduling
    tensor/state lifetime
    cleanup and single-stream fallback

Connector stream adapter
  CAMP2P adapter:
    recv = a2e
    send = e2a
    transfer state = CAMP2PTransferState

  CAMAsync adapter:
    recv = async_dispatch_recv
    send = async_combine_send
    transfer state = AFDAsyncTransferState
    optional Attention-side dispatch/combine phases

The shared executor should consume explicit work items and phase callbacks or a typed adapter contract. It must not inspect connector internals with pervasive getattr/hasattr, and it must not add mutable global stream state.

The first implementation should prioritize the FFN pipeline because it maps directly to #68:

receive work N+1 on recv stream
        overlaps
compute work N on compute stream
        overlaps
send work N-1 on send stream

Each edge must be represented by an explicit event:

• receive complete before compute reads routed/shared activations and quantization scales;
• compute complete before send reads routed/shared outputs;
• send complete before a slot's buffers or transfer state are reused;
• layer/stage identity and CAM pending FIFO ordering remain correct.

After FFN-side correctness is established, the same framework may be extended to the CAMAsync Attention side for async_dispatch_send/async_combine_recv overlap. Attention-side support should be a separate reviewed phase because its cross-layer and two-microbatch dependency graph differs from the FFN work-item loop.

Configuration and capability gating

Define one typed multistream configuration contract shared by supported NPU connectors. Do not restore the recently removed ambiguous legacy switches (is_multistream, is_attn_multistream, is_ffn_multistream, or untyped multistream_info) without an explicit compatibility decision.

The final configuration should:

• default to the existing single-stream behavior;
• allow role/phase enablement only where implemented;
• validate connector and operator capabilities during initialization;
• bound the number of in-flight work items and streams;
• fail early for unsupported graph, topology, quantization, or async-MoE combinations;
• use the same common schema for CAMP2P and CAMAsync, with small connector-specific extensions only when required.

vLLM compatibility and extension points

Preferred extension points:

• plugin-owned AFDNPUFFNModelRunner connector-driven and control-plane-driven execution;
• plugin-owned NPU FFN worker loop;
• plugin-owned CAMP2P and CAMAsync connector adapters;
• plugin-owned CAMAsync Attention/model scheduling after the FFN phase is validated.

Compat shim needed: only for pinned vLLM v0.19.1 / vLLM-Ascend v0.19.1rc1 stream or forward-context contracts that cannot be accessed through stable interfaces.

Monkey patch needed: none expected. Do not modify vLLM or vLLM-Ascend source trees.

Phased implementation plan

Phase 0: design and operator-contract validation

• Document CAMP2P and CAMAsync phase DAGs, ownership, event edges, buffer reuse, error propagation, and shutdown.
• Run focused probes against afd_ascend and umdk_cam_op_lib to verify current-stream behavior and concurrent communicator/buffer safety.
• Decide the shared adapter/config API and the supported initial feature matrix.
• Review the design jointly with #68 before implementation.

Phase 1: common runtime and single-stream parity

• Add the shared executor, typed work slots, event pool, adapter interface, and lifecycle management.
• Route both CAMP2P and CAMAsync through it in single-stream compatibility mode.
• Prove identical call ordering and outputs before enabling overlap.

Phase 2: FFN-side multi-stream

• Enable receive/compute/send overlap for CAMP2P (#68) and CAMAsync using the same executor.
• Replace per-step device-wide synchronization with the narrow event waits required by dependencies and shutdown.
• Keep a runtime switch for immediate fallback to serialized execution.

Phase 3: CAMAsync Attention-side evaluation

• Model the dispatch/compute/combine dependency graph across layers and async-MoE stages.
• Reuse the same stream/event/work-slot primitives.
• Enable only the combinations validated by tests and operator probes.

Phase 4: tuning and cleanup

• Tune in-flight depth and event reuse from measurements.
• Remove temporary connector-local scheduling code.
• Document the final support matrix and operational controls.

Validation plan

• CPU-safe tests for configuration parsing, capability gating, phase DAG construction, slot reuse, exception cleanup, shutdown, and serialized fallback.
• Mock-stream tests asserting exact event record/wait ordering and proving buffers are not reused before send completion.
• Connector tests preserving CAMP2P transfer state and CAMAsync routing/dynamic-quant metadata across in-flight work.
• NPU correctness tests comparing single-stream and multi-stream outputs for CAMP2P and CAMAsync.
• CAMAsync E2E coverage for prefill, including supported async-MoE ubatching and topology combinations from the Phase 0 matrix.
• Stress tests with skewed token routing, zero routed tokens on an FFN rank, multiple layers, repeated stages, and shutdown during blocked receive.
• Profiling traces showing actual overlap among receive communication, FFN computation, and send communication; operator names and stream IDs must make overlap auditable.
• Performance comparison in communication-bound and FFN-compute-bound workloads, reporting throughput and per-layer latency without setting a pass threshold until the operator-contract probe is complete.
• Regression coverage for default single-stream CAMP2P and CAMAsync behavior.

Acceptance criteria

• A reviewed design explicitly maps the common framework to both this issue and #68 before multi-stream implementation begins.
• CAMP2P and CAMAsync use the same concrete stream executor, event pool, work-slot lifecycle, configuration model, and fallback path; the shared layer is exercised by both, not merely prepared for future reuse.
• Connector-specific code is limited to typed operator adapters and transfer-state semantics.
• CAMAsync FFN receive/compute/send overlap is visible in an NPU profiling trace and produces output equivalent to the serialized path.
• No device-wide synchronization remains in the steady-state multi-stream loop unless Phase 0 proves it is required by an external operator; any such synchronization is isolated and documented.
• Tensor buffers, routing metadata, quantization scales, transfer states, and communicator resources remain valid until their dependent event completes.
• Unsupported feature combinations fail during initialization, and the default remains safe single-stream execution.
• No vLLM or vLLM-Ascend source modification is required.

Before submitting

• I searched existing issues and RFCs.
• I identified this as plugin-owned NPU scheduling and connector-adapter work, with a versioned compat helper only if required by the pinned runtime.