vt::Conv2d has no CUDA provider, so the shared 2-D convolution seam cannot run on a GPU
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 423
- Forks
- 53
- Avg merge
- 20h 26m
- Merged PRs (30d)
- 310
Description
Row: KERNEL-VT-CONV2D-CUDA
vt::Conv2d (include/vt/ops.h:4428) has exactly ONE provider in the tree:
$ grep -rn "OpId::kConv2d" src/ include/
src/vt/op_provider.cpp:566: case OpId::kConv2d:
src/vt/ops.cpp:3282: reinterpret_cast<Conv2dFn>(GetOp(OpId::kConv2d, q.device.type))(q, out, x, weight, bias, args);
src/vt/cpu/cpu_conv2d.cpp:111: RegisterOp(OpId::kConv2d, DeviceType::kCPU,
src/vt/cpu/cpu_conv2d.cpp:111 is the CPU registration and there is no CUDA
one, so vt::Conv2d on a CUDA queue resolves no op. Any model that routes a
2-D convolution through the shared seam therefore ships a capability that
cannot run on the only device class that could serve it.
Why this is being filed now
dots3-note W7a (#2703) ports the dots speech encoder, whose stem is three
Conv2d(kernel=3, stride=2) layers (nvidia/audio_encoder.py:466-474 @
9035151d6). It composes them as host im2col + vt::MatmulBT and records ONE
EXACT TRACKED EXCEPTION to the kConv2d seam, with this issue as the reason.
The in-tree precedent is include/vllm/model_executor/models/whisper_audio.h:33,
which composes its Conv1d stem the same way and says "no new CUDA kernel".
The exception is the right call for that slice — a stem routed through
kConv2d would fault on the queue every real deployment uses — but the gap
outlives it. It is a vt gap, not a model gap.
What closing it looks like
A CUDA provider for OpId::kConv2d matching the CPU one's contract
(src/vt/cpu/cpu_conv2d.cpp), gated the way the other CUDA op ports are: a
red-first CPU/CUDA agreement test over the shapes the registered callers use,
plus the arch gate check-cuda-op-arch-gate already applies. Then each im2col
composition that exists only because of this gap can be reconsidered on a
measurement rather than on availability.
Not in scope here
vt::Conv3d (ops.h:4475) and vt::Conv1d (ops.h:4609) have their own
provider tables and are not measured by this issue.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with include/vt/ops.h:4428, src/vt/cpu/cpu_conv2d.cpp:111, and the dispatch in src/vt/ops.cpp:3282 to understand the Conv2d contract and provider registration. Run the existing check-cuda-op-arch-gate and inspect the red-first CPU/CUDA agreement coverage for registered caller shapes. Done means a CUDA OpId::kConv2d provider is registered, passes the agreement test, and satisfies the architecture gate.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- backend, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 66/100