huggingface / huggingface/course

Suggestion: Improve token–NER display with defaultdict for readability

Open
#1,012 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
MDX
Stars
4.2k
Forks
1.4k
Avg merge
13m
Merged PRs (30d)
1

Description

In chapter 7, [Token Classification](https://huggingface.co/learn/llm-course/chapter7/2#the-conll-2003-dataset)
the original way to decode labels is not much clear and feels overwhelming for beginners and might distract from the original task to just decode labels.

Original way:
```python3
words = raw_datasets["train"][0]["tokens"]
labels = raw_datasets["train"][0]["ner_tags"]
line1 = ""
line2 = ""
for word, label in zip(words, labels):
full_label = label_names[label]
max_length = max(len(word), len(full_label))
line1 += word + " " * (max_length - len(word) + 1)
line2 += full_label + " " * (max_length - len(full_label) + 1)

print(line1)
print(line2)
```

here we are manually doing it.

A better way in my opinion is to use **defaultdict**

My implementation:
```python3
from collections import defaultdict
words = raw_datasets["train"][4]["tokens"]
labels = raw_datasets["train"][4]["ner_tags"]

word2label = defaultdict(str)

for word, label in zip(words, labels):
word2label[word] = label_names[label]
for k, v in word2label.items():
print(k,'-'*(16-len(k)), v)
```

Which gives a much more friendly experience:

Image

I'm opening this issue to confirm if you'd be open to this improvement.
If it's approved, I plan to submit a PR myself.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.