pytorch / pytorch/vision

Windows C++ library build issues

Open
#3,086 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

module: c++ frontend windows
Dominant language
Python
Stars
17.9k
Forks
7.3k
Avg merge
1d 15h
Merged PRs (30d)
13

Description

🐛 Bug

Thanks for your work in putting up this great software. PyTorch and TorchVision are two amazing tools.

I encountered two different issues when trying to build the C++ library on Windows:

  1. The debug build tries to link to the debug python library which most people do not have.
  2. The operators registration code was not included in consuming projects.

To Reproduce

Steps to reproduce the behavior:

  1. Download the latest LibTorch builds
  2. Build and install the vision C++ library with CMake by giving it the proper CMake arguments, e.g. Torch_DIR, etc.
  3. Build a small exectuable that lists the registered ops like this:

Credits to @zsef123 for the code below

#include <iostream>

#include <torch/torch.h>
#include "vision.h"

int main(int argc, char* argv[]) {
    auto& ops = torch::jit::getAllOperators();
    std::cout << "torch jit operators\n";
    for (auto& op: ops) {
        auto& name = op->schema().name();
        if (name.find("torchvision") != std::string::npos)
            std::cout << "op : " << op->schema().name() << "\n";
    }
    std::cout << "\n";
    return 0;
}

Expected behavior

A debug build should be possible since it is almost always required on Windows.
Operators should register automatically in consuming binaries.

Environment

  • PyTorch / torchvision Version (e.g., 1.7.0 / master):
  • OS: Windows 10:
  • Installed LibTorch from pytorch.org
  • cmake --build .
  • Python version: 3.8
  • CUDA/cuDNN version: 10.2

Additional context

  • The first issue is easily fixed by ignoring the python debug library by adding the /NODEFAULTLIB:python38_d.lib linker flag.
  • The second issue is in fact multiple issues. First, if I am not mistaken, since the vision.h file is never included in the vision.cpp the cuda_version definition is considered as a different symbol from the declaration in vision.h. An easy fix, is to add VISION_API in front the definition. However, this is more elegantly fixed by including the vision.h header and changing it so it builds correctly. Second, even with this symbol included in the consuming library, the linker strips the operators registration symbols in a Release build. This is fixed by using the /OPT:NOREF linker flags.

I will submit a PR that fixes all these above issues.

cc @peterjc123 @nbcsm @guyang3532 @maxluk @gunandrose4u @smartcat2010 @mszhanyi

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the Windows C++ build with CMake and the stated Torch_DIR and LibTorch setup. Inspect vision.cpp, vision.h, and the linker settings for the Python debug library and operator registration. Done means debug builds no longer require python38_d.lib and operators register in consuming Debug and Release binaries.

Written by the indexing model from the issue text.

Assessment

Tech stack
cmake, cpp, python
Domain
build-system, computer-vision
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.