huggingface / huggingface/course
Transformers, what can they do?
- Dominant language
- MDX
- Stars
- 4.2k
- Forks
- 1.4k
- Avg merge
- 13m
- Merged PRs (30d)
- 1
Description
Hello,
Going via the training.
Some small ideas for improvements.
#######################
Transformers, what can they do?
https://huggingface.co/learn/nlp-course/en/chapter1/3
A)
Current code sample
is incomplete
from transformers import pipeline
classifier = pipeline("sentiment-analysis")
classifier("I've been waiting for a HuggingFace course my whole life.")
CORRECT COULD BE
B)
from transformers import pipeline
classifier = pipeline("sentiment-analysis")
result = classifier("I've been waiting for a HuggingFace course my whole life.")
print(result)
C)
Even better could be
====================
import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3' # Suppresses TensorFlow logs
os.environ['TF_ENABLE_ONEDNN_OPTS'] = '0' # Disables oneDNN custom operations
from transformers import pipeline, AutoTokenizer, AutoModelForSequenceClassification
import warnings
import torch
import tensorflow as tf
tf.get_logger().setLevel('ERROR')
# Set environment variable to disable oneDNN custom operations warning (specific to TensorFlow)
os.environ['TF_ENABLE_ONEDNN_OPTS'] = '0'
# Suppress warnings
warnings.filterwarnings('ignore', category=DeprecationWarning)
# Check if a GPU is available
device = 0 if torch.cuda.is_available() else -1
# Load the tokenizer with the clean_up_tokenization_spaces parameter set
tokenizer = AutoTokenizer.from_pretrained(
"distilbert/distilbert-base-uncased-finetuned-sst-2-english",
clean_up_tokenization_spaces=True
)
# Load the model in PyTorch
model = AutoModelForSequenceClassification.from_pretrained(
"distilbert/distilbert-base-uncased-finetuned-sst-2-english"
)
# Initialize the sentiment-analysis pipeline with the custom tokenizer and PyTorch model
classifier = pipeline(
"sentiment-analysis",
model=model,
framework='pt', # Use PyTorch
tokenizer=tokenizer,
device=device # Use GPU if available, otherwise use CPU
)
result = classifier(
["I've been waiting for a HuggingFace course my whole life.", "I hate this so much!"]
)
print(result)
######################
https://huggingface.co/learn/nlp-course/en/chapter8/5
transformers-cli env
- `transformers` version: 4.44.0
- Platform: Windows-10-10.0.22631-SP0
- Python version: 3.11.9
- Huggingface_hub version: 0.24.5
- Safetensors version: 0.4.4
- Accelerate version: 0.33.0
- Accelerate config: not found
- PyTorch version (GPU?): 2.4.0+cu118 (True)
- Tensorflow version (GPU?): 2.17.0 (False)
- Flax version (CPU?/GPU?/TPU?): 0.7.0 (cpu)
- Jax version: 0.4.13
- JaxLib version: 0.4.13
- Using distributed or parallel set-up in the script?:
- Using GPU in script?:
- GPU type: NVIDIA GeForce RTX 3060 Laptop GPU
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.