Make the C++ backend of the torchvision wheel usable for C++ development

Open

#9,042 opened on Apr 29, 2025 5 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language: Python
Stars: 17.9k
Forks: 7.3k
Avg merge: 1d 15h
Merged PRs (30d): 13

Description

🚀 The feature

Currently, the torchvision wheel packages the C++ DSO as _C.so for python bindings.

We'd like the python wheel to have the C++ backend be standalone, so it can be extracted/used by C++ applications, like is done today for the PyTorch wheels.

This means:

export DSO as libtorchvision.so instead of _C.so
do not hardlink libtorchvision.so against libtorch_python.so.
- maybe _C.so is kept for symbols that require libtorch_python.so ?
export cpp headers
export CMake configs

Motivation, pitch

C++ developers can currently use the distributed PyTorch wheels to develop C++ native applications against libtorch, as libraries, headers, and cmake configs are available in the wheels.

C++ developers who also need to use torchvision cannot leverage the standard vision wheel the same way even though all C++ symbols are available in _C.so. Instead, they must build libtorchvision C++ from source which is more cumbersome, requires extra dev packages to be installed, especially for cuda support.

Additional context

see ld links for torchvision 0.22.0+cu128 (wheel)

libc.so.6
libc10.so
libc10_cuda.so
libcudart.so.12
libdl.so.2
libgcc_s.so.1
libm.so.6
libpthread.so.0
librt.so.1
libstdc++.so.6
libtorch.so
libtorch_cpu.so
libtorch_cuda.so
libtorch_python.so # requires python
linux-vdso.so.1

see ld links for c++ source build of torchvision

no link against libtorch_python.so

libc.so.6
libc10.so
libc10_cuda.so
libcudart.so.12
libdl.so.2
libgcc_s.so.1
libm.so.6
libpthread.so.0
librt.so.1
libstdc++.so.6
libtorch.so
libtorch_cpu.so
libtorch_cuda.so
linux-vdso.so.1

example of a cpp torchvision installation with files needed for C++ development

The install tree below can be imported for building with CMake with:

cmake ... -D TorchVision_ROOT="$torch_vision_install_dir"  # Or add to CMAKE_PREFIX_PATH

find_package(TorchVision)

├── include
│   └── torchvision
│       ├── io
│       │   └── image
│       │       ├── cpu
│       │       │   ├── common_jpeg.cpp
│       │       │   ├── common_jpeg.h
│       │       │   ├── common_png.h
│       │       │   ├── decode_gif.cpp
│       │       │   ├── decode_gif.h
│       │       │   ├── decode_image.cpp
│       │       │   ├── decode_image.h
│       │       │   ├── decode_jpeg.cpp
│       │       │   ├── decode_jpeg.h
│       │       │   ├── decode_png.cpp
│       │       │   ├── decode_png.h
│       │       │   ├── encode_jpeg.cpp
│       │       │   ├── encode_jpeg.h
│       │       │   ├── encode_png.cpp
│       │       │   ├── encode_png.h
│       │       │   ├── exif.h
│       │       │   ├── giflib
│       │       │   │   ├── dgif_lib.c
│       │       │   │   ├── gif_hash.c
│       │       │   │   ├── gif_hash.h
│       │       │   │   ├── gif_lib.h
│       │       │   │   ├── gif_lib_private.h
│       │       │   │   ├── gifalloc.c
│       │       │   │   └── openbsd-reallocarray.c
│       │       │   ├── read_write_file.cpp
│       │       │   └── read_write_file.h
│       │       ├── cuda
│       │       │   ├── decode_jpeg_cuda.cpp
│       │       │   ├── encode_decode_jpegs_cuda.h
│       │       │   ├── encode_jpegs_cuda.cpp
│       │       │   └── encode_jpegs_cuda.h
│       │       ├── image.cpp
│       │       ├── image.h
│       │       └── image_read_mode.h
│       ├── macros.h
│       ├── ops
│       │   ├── autocast
│       │   │   ├── deform_conv2d_kernel.cpp
│       │   │   ├── nms_kernel.cpp
│       │   │   ├── ps_roi_align_kernel.cpp
│       │   │   ├── ps_roi_pool_kernel.cpp
│       │   │   ├── roi_align_kernel.cpp
│       │   │   └── roi_pool_kernel.cpp
│       │   ├── autograd
│       │   │   ├── deform_conv2d_kernel.cpp
│       │   │   ├── ps_roi_align_kernel.cpp
│       │   │   ├── ps_roi_pool_kernel.cpp
│       │   │   ├── roi_align_kernel.cpp
│       │   │   └── roi_pool_kernel.cpp
│       │   ├── cpu
│       │   │   ├── deform_conv2d_kernel.cpp
│       │   │   ├── nms_kernel.cpp
│       │   │   ├── ps_roi_align_kernel.cpp
│       │   │   ├── ps_roi_pool_kernel.cpp
│       │   │   ├── roi_align_common.h
│       │   │   ├── roi_align_kernel.cpp
│       │   │   └── roi_pool_kernel.cpp
│       │   ├── cuda
│       │   │   ├── cuda_helpers.h
│       │   │   ├── deform_conv2d_kernel.cu
│       │   │   ├── nms_kernel.cu
│       │   │   ├── ps_roi_align_kernel.cu
│       │   │   ├── ps_roi_pool_kernel.cu
│       │   │   ├── roi_align_kernel.cu
│       │   │   └── roi_pool_kernel.cu
│       │   ├── deform_conv2d.cpp
│       │   ├── deform_conv2d.h
│       │   ├── nms.cpp
│       │   ├── nms.h
│       │   ├── ops.h
│       │   ├── ps_roi_align.cpp
│       │   ├── ps_roi_align.h
│       │   ├── ps_roi_pool.cpp
│       │   ├── ps_roi_pool.h
│       │   ├── roi_align.cpp
│       │   ├── roi_align.h
│       │   ├── roi_pool.cpp
│       │   └── roi_pool.h
│       ├── vision.cpp
│       └── vision.h
├── lib
│   └── libtorchvision.so
└── share
    └── cmake
        └── TorchVision
            ├── TorchVisionConfig.cmake
            ├── TorchVisionConfigVersion.cmake
            ├── TorchVisionTargets-noconfig.cmake
            └── TorchVisionTargets.cmake

How libtorch C++ can be used today with C++ CMake projects

torch_install=$(python3 -c "import torch; print(torch.__path__[0])")
cmake ... -D Torch_ROOT="$torch_install"  # Or add to CMAKE_PREFIX_PATH

find_package(Torch)

> ls $torch_install/include
ATen/      dnnl_config.h  dnnl_sycl_types.h     fp16.h    psimd.h         tensorpipe/
c10/       dnnl_debug.h   dnnl_threadpool.h     fxdiv.h   pthreadpool.h   THC/
caffe2/    dnnl.h         dnnl_types.h          kineto/   pybind11/       torch/
clog.h     dnnl_ocl.h     dnnl_version.h        libshm.h  qnnpack_func.h  xnnpack.h
cpuinfo.h  dnnl_sycl.h    experiments-config.h  nnpack.h  sleef.h

> ls $torch_install/include
ATen/      dnnl_config.h  dnnl_sycl_types.h     fp16.h    psimd.h         tensorpipe/
c10/       dnnl_debug.h   dnnl_threadpool.h     fxdiv.h   pthreadpool.h   THC/
caffe2/    dnnl.h         dnnl_types.h          kineto/   pybind11/       torch/
clog.h     dnnl_ocl.h     dnnl_version.h        libshm.h  qnnpack_func.h  xnnpack.h
cpuinfo.h  dnnl_sycl.h    experiments-config.h  nnpack.h  sleef.h

> ls $torch_install/share/cmake/Torch/
TorchConfig.cmake  TorchConfigVersion.cmake

> ls $torch_install/lib
libarm_compute_graph.so*                 libcudnn_ops.so.9*       libnvpl_blas_lp64_gomp.so.0*
libarm_compute.so*                       libcudnn.so.9*           libnvpl_lapack_core.so.0
libc10_cuda.so*                          libcufft.so.11*          libnvpl_lapack_lp64_gomp.so.0
libc10.so*                               libcufile_rdma.so.1*     libnvrtc-builtins.so.12.8*
libcaffe2_nvrtc.so*                      libcufile.so.0*          libnvrtc.so.12*
libcublasLt.so.12*                       libcupti.so.12*          libnvToolsExt.so.1*
libcublas.so.12*                         libcurand.so.10*         libshm.so*
libcudart.so.12*                         libcusolver.so.11*       libtorch_cpu.so*
libcudnn_adv.so.9*                       libcusparseLt.so.0       libtorch_cuda_linalg.so*
libcudnn_cnn.so.9*                       libcusparse.so.12*       libtorch_cuda.so*
libcudnn_engines_precompiled.so.9*       libgfortran.so.5*        libtorch_global_deps.so*
libcudnn_engines_runtime_compiled.so.9*  libgomp.so.1*            libtorch_python.so*
libcudnn_graph.so.9*                     libnvJitLink.so.12*      libtorch.so*
libcudnn_heuristic.so.9*                 libnvpl_blas_core.so.0*

Contributor guide

Open the contributing guide

First steps

Read the whole issue, then the project's contributing guide.
Comment on the issue to say you are picking it up — it saves two people doing the same work.
Fork the repository and make your change on a branch.
Open a pull request that references the issue number.

Research direction

Start by tracing the wheel packaging and CMake export entry points that produce the current _C.so; the issue provides no source files or tests to begin with. Done means the wheel exposes lib/libtorchvision.so, include/torchvision, and share/cmake/TorchVision, without requiring libtorch_python.so for the standalone library.

Written by the indexing model from the issue text.

Assessment

Tech stack: cmake, cpp, python
Domain: build-system, developer-experience
Issue type: Feature
Difficulty: 5/5
Estimated time: Over a week
Activity status: Stale
Clarity: Needs clarification
Newbie friendliness: 25/100