huggingface / huggingface/candle

Implement Conv3D support for compatibility with Qwen-VL and similar models

Open
#2,997 1 comment 3 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
21k
Forks
1.8k
Avg merge
16h 42m
Merged PRs (30d)
25

Description

Several vision-language models such as Qwen-VL and its variants make use of 3D convolution layers (Conv3D) in their architecture, especially for handling video or temporal spatial data. Currently, Candle does not support Conv3D operations, which makes it impossible to run or port such models natively.

In order to support these models and ensure broader compatibility with existing open-source architectures, it would be beneficial to implement Conv3D in Candle as a fundamental operation.

This will enable:

- Native execution of Qwen-VL-style models
- Proper handling of video or spatio-temporal data inputs
- Compatibility with pretrained weights relying on Conv3D layers

Looking forward to discussion and suggestions on how best to approach this implementation.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.