EleutherAI / EleutherAI/knowledge-neurons

grad error using KnowledgeNeurons with model_name = "gpt2"

Open
#2 5 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
159
Forks
19
PR merge metrics
No merged PRs in 30d

Description

When I initialize KnowledgeNeurons with model_name = 'gpt2', I get an RuntimeError when trying to run kn.get_refined_neurons()

The following snippet would be able to reproduce the error on a colab notebook:
```
!pip install knowledge-neurons
!nvidia-smi
from knowledge_neurons import (
KnowledgeNeurons,
initialize_model_and_tokenizer,
model_type,
)
import random
import torch
import torch.nn.functional as F

# setup model, tokenizer + kn class
MODEL_NAME = "gpt2"
model, tokenizer = initialize_model_and_tokenizer(MODEL_NAME)
kn = KnowledgeNeurons(model, tokenizer,model_type(MODEL_NAME))

TEXT = "Sarah was visiting [MASK], the capital of france"
GROUND_TRUTH = "paris"
BATCH_SIZE = 10
STEPS = 20

ENG_TEXTS = [
"Sarah was visiting [MASK], the capital of france",
"The capital of france is [MASK]",
"[MASK] is the capital of france",
"France's capital [MASK] is a hotspot for romantic vacations",
"The eiffel tower is situated in [MASK]",
"[MASK] is the most populous city in france",
"[MASK], france's capital, is one of the most popular tourist destinations in the world",
]
FRENCH_TEXTS = [
"Sarah visitait [MASK], la capitale de la france",
"La capitale de la france est [MASK]",
"[MASK] est la capitale de la france",
"La capitale de la France [MASK] est un haut lieu des vacances romantiques",
"La tour eiffel est située à [MASK]",
"[MASK] est la ville la plus peuplée de france",
"[MASK], la capitale de la france, est l'une des destinations touristiques les plus prisées au monde",
]
TEXTS = ENG_TEXTS + FRENCH_TEXTS

refined_neurons_eng = kn.get_refined_neurons(
ENG_TEXTS,
GROUND_TRUTH,
p=P,
batch_size=BATCH_SIZE,
steps=STEPS,
)
refined_neurons_fr = kn.get_refined_neurons(
FRENCH_TEXTS,
GROUND_TRUTH,
p=P,
batch_size=BATCH_SIZE,
steps=STEPS,
)
refined_neurons = kn.get_refined_neurons(
TEXTS,
GROUND_TRUTH,
p=P,
batch_size=BATCH_SIZE,
steps=STEPS,
)
```

Given below is the full traceback:

```Getting coarse neurons for each prompt...: 0%| | 0/7 [00:00 in ()
45 p=P,
46 batch_size=BATCH_SIZE,
---> 47 steps=STEPS,
48 )
49 refined_neurons_fr = kn.get_refined_neurons(

5 frames
/usr/local/lib/python3.7/dist-packages/torch/autograd/__init__.py in _make_grads(outputs, grads)
49 if out.requires_grad:
50 if out.numel() != 1:
---> 51 raise RuntimeError("grad can be implicitly created only for scalar outputs")
52 new_grads.append(torch.ones_like(out, memory_format=torch.preserve_format))
53 else:

RuntimeError: grad can be implicitly created only for scalar outputs
```
Grad Error when initializing for the "gpt2" model
@StellaAthena @sdtblck also what should be the input and the target for "gpt2" model given it's autoregressive mechanism??

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.