Model instantiation without loading from disk

🚀 The feature

So far, the only way to use a model from torchvision is through loading a jit checkpoint from the disk like so:

#include <torch/script.h>

#include <iostream>
#include <memory>

int main(int argc, const char* argv[]) {
  if (argc != 2) {
    std::cerr << "usage: example-app <path-to-exported-script-module>\n";
    return -1;
  }

  // Deserialize the ScriptModule from a file using torch::jit::load().
  std::shared_ptr<torch::jit::script::Module> module = torch::jit::load(argv[1]);

  assert(module != nullptr);
  std::cout << "ok\n";
}

The feature that I would like to propose is to purge away the need to have a precompiled jit file and integrate a methodology in the C++ PyTorch frontend that can easily instantiate any torchvision.models file as easily as in Python. For example:

#include <torch/script.h>

#include <iostream>
#include <memory>

int main(int argc, const char* argv[]) {
  std::shared_ptr<torch::jit::script::Module> module = torch::jit::load(torchvision::models::resnet50);

  assert(module != nullptr);
  std::cout << "ok\n";
}

Motivation, pitch

There shouldn't be any dependencies between the Python frontend and the C++ frontend. Specifically, there are projects that leverage the C++ PyTorch API solely, and in that case, the developers have to invoke every time a Python script before the utilization of their framework just to create an instance of the desired model from torchvision.models to then use their framework. This is a timely process, particularly if there is frequent model change at runtime.

Specific use case:
I am building a framework that is connected with Torch TensorRT and utilizes NVIDIA NVDLAs of Jetson boards. However, every time I query my framework for some workload, I have to first use Python and compile a jit instance to later load in my framework. This creates a huge overhead and since disk operations are most timely, it defeats the whole purpose of using C++ to accelerate the process.

Start by reviewing the C++ frontend API around torch::jit::load and the torchvision.models entry points described in the issue. Determine how model definitions and weights are currently exposed, then define the scope and acceptance criteria for creating a model without a disk checkpoint; no specific files or tests are named.