google-research / google-research/pix2struct

how to get Confidence score or probability score of Pix2struct VQA model

Open
#36 3 comments 1 reaction 0 assignees View on GitHub
Dominant language
Python
Stars
690
Forks
61
PR merge metrics
No merged PRs in 30d

Description

How do we get the confidence score of predictions for pix2struct model as mentioned in the below code snippet in pred[0], how do we get the prediction scores or probability scores?

```
FILENAME = "XXX.pdf"
PAGE_NO = 1
DEVICE = "cuda" if torch.cuda.is_available() else "cpu"

model = psg.from_pretrained("google/pix2struct-docvqa-base").to(DEVICE)
processor = psp.from_pretrained("google/pix2struct-docvqa-large")

def convert_pdf_to_image(filename, page_no):
return convert_from_path(filename)[page_no-1]

image = convert_pdf_to_image(FILENAME, PAGE_NO)

inputs = processor(images=[image for _ in range(len(questions))], text=questions, return_tensors="pt").to(DEVICE)

pred=model.generate(**inputs)
print(processor.decode(pred[0],skip_special_tokens=True))
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.