GPT-2 Generate multiple words using past

ort_inputs = dict((sess.get_inputs()[i].name, np.expand_dims(input, 0)) for i, input in enumerate(inputs))
res = sess.run(None, ort_inputs)
In pytorch i iterate over a for loop to generate the multiple words.How can the same be done with onnxruntime.

I was able to do some research and find out we need to use torch script for loops
import torch.nn.functional as F
@torch.jit.script
def loop(input_ids, token_type_ids):
out = torch.jit.annotate(List[int],[])
output = torch.jit.annotate(List[Tuple[Tensor, Tensor]], [])
past = torch.jit.annotate(List[Tensor, Tensor,Tensor,Tensor,Tensor,Tensor,Tensor,Tensor,Tensor,Tensor,Tensor,Tensor], [])
for i in range(10):
logits,past = model(input_ids,token_type_ids)
logits = output[0, -1, :] / 1.2
probs = F.softmax(logits, dim=-1)
prev = torch.topk(probs, 1)[1]
print("here")
if prev.item() in [50257, 50258, 50259, 50260, 50261, 50262]:
break
out.append(prev.item())
input_ids = prev.unsqueeze(0)
token_type_ids = torch.tensor([50261]).unsqueeze(0)
print(out)
return out

class LoopModel2(torch.nn.Module):
def forward(self, input_ids, token_type_ids):
return loop(input_ids, token_type_ids)

But the model returns a tuple object which is not supported by . torch script is there any work around for this

Start with the ONNX Runtime session call using ort_inputs and sess.run, then compare it with the TorchScript loop that iteratively updates input_ids and past. Investigate how the model's tuple return is represented and how past values are fed into subsequent steps; done means generating multiple words with ONNX Runtime as in the PyTorch example.