[Documentation]: Overview of C# Workflow for consuming "Augmented" Onnx model with Custom Operators

@natke

Onnxruntime provides contributed operators

NLP use case

Step1

In PyTorch, creates an "augmented" onnx model which includes pre- and post- processing (Contributed Operators).

Build an augmented ONNX model with BERT pre and processing.

from pathlib import Path
import torch
from transformers import AutoTokenizer
import onnx
from onnxruntime_extensions import pnp

# The fine-tuned HuggingFace model is exported to ONNX in the code snippet above
model_name = "distilbert-base-uncased-finetuned-sst-2-english"
model_path = Path(model_name + ".onnx")

# mapping the BertTokenizer outputs into the onnx model inputs
def map_token_output(input_ids, attention_mask, token_type_ids):
    return input_ids.unsqueeze(0), token_type_ids.unsqueeze(0), attention_mask.unsqueeze(0)

# Post process the start and end logits
def post_process(*pred):
    output = torch.argmax(pred[0])
    return output

tokenizer = AutoTokenizer.from_pretrained(model_name)
bert_tokenizer = pnp.PreHuggingFaceBert(hf_tok=tokenizer)
bert_model = onnx.load_model(str(model_path))

augmented_model = pnp.SequentialProcessingModule(bert_tokenizer, map_token_output,
                                                 bert_model, post_process)

test_input = ["This is s test sentence"]

# create the final onnx model which includes pre- and post- processing.
augmented_model = pnp.export(augmented_model,
                             test_input,
                             opset_version=12,
                             input_names=['input'],
                             output_names=['output'],
                             output_path=model_name + '-aug.onnx',
                             dynamic_axes={'input': [0], 'output': [0]})

Step2

Follow WIP [Documentation]: C# Workflow for consuming "Augmented" Onnx model with Custom Operators to create Custom_Op_Library.dll with the pre- and post- processing (Contributed Operators).

Step3

In CSharp, registered Custom_Op_Library.dll with the pre- and post- processing (Contributed Operators) before consuming the "augmented" Onnx model created in Step1

Start with the linked WIP issue 11657 and the existing ContribOperators.md documentation. Trace the three-step workflow shown here: create the augmented model in Python, build the custom operator library, and register it from C# before consuming the model. Done means the documentation clearly covers this end-to-end workflow.