[WIP] Running longformer on TPU using pytorch/xla

We try running wrapped longformer model under colab TPU and got the following errors:

Tvm binary not found. Compiling ...
Exception in device=TPU:0: cannot import name 'nvcc'
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/torch_xla/distributed/xla_multiprocessing.py", line 119, in _start_fn
fn(gindex, *args)
File "", line 66, in _mp_fn
fitter.fit(train_loader, validation_loader)
File "", line 47, in fit
losses, final_scores = self.train_one_epoch(para_loader.per_device_loader(self.device))
File "", line 120, in train_one_epoch
outputs = self.model(inputs, attention_masks)
File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 558, in __call__
result = self.forward(*input, **kwargs)
File "", line 26, in forward
seq_x, _ = self.backbone(input_ids=input_ids, attention_mask=attention_masks)
File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 558, in __call__
result = self.forward(*input, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/transformers/modeling_bert.py", line 790, in forward
....

Anyway to work around this error will be appreciated.
Thanks.