Zero shot classification inference

Hi

I need help with using deepspeed for transformers zero shot classification pipeline.

My dataset has 500K sentences and 52 labels.

I tried editing the gpt generation example but I am not sure if I did it right.

import os
import deepspeed
import torch
import transformers
from transformers import pipeline

local_rank = int(os.getenv('LOCAL_RANK', '0'))
world_size = int(os.getenv('WORLD_SIZE', '1'))
classifier = pipeline("zero-shot-classification", model="typeform/distilbert-base-uncased-mnli", device=0)



classifier.model = deepspeed.init_inference(classifier.model,
                                           mp_size=world_size,
                                           dtype=torch.float,
                                           replace_method='auto')

res =  classifier(prod_name_lst[:1000], tag_values)
if torch.distributed.get_rank() == 0:
    print(res)

Kindly help with an example.

Thanks,
Subham

Start by reviewing the provided Python snippet and the GPT generation example it references, then verify how the zero-shot-classification pipeline is intended to work with DeepSpeed inference. Done would be a clear, tested example for the stated dataset and label setup, but the issue does not name repository files or tests.