← Back to Projects
AI Development
😀

Hugging Face Transformers

State-of-the-art Machine Learning for PyTorch, TensorFlow, and JAX. A powerful library for working with pre-trained language models.

Intermediate NLP Computer Vision Framework
🐙 GitHub Repository 🌐 Official Website

Alternative To

  • • OpenAI API
  • • Google Cloud NLP
  • • Amazon Comprehend

Difficulty Level

Intermediate

Requires some technical experience. Moderate setup complexity.

Overview

Hugging Face Transformers is a state-of-the-art natural language processing library that provides thousands of pre-trained models to perform tasks on texts such as classification, information extraction, question answering, summarization, translation, text generation, and more in over 100 languages. It also provides APIs and tools to easily download and train state-of-the-art pre-trained models.

Why Hugging Face Transformers for AI Development?

Hugging Face Transformers has become the go-to library for NLP tasks because it offers:

  • Access to thousands of pre-trained models for various NLP tasks
  • Support for PyTorch, TensorFlow, and JAX frameworks
  • Easy-to-use APIs for fine-tuning models on custom datasets
  • Optimized performance for both research and production environments
  • Active community and regular updates with the latest NLP advancements

System Requirements

  • CPU: 4+ cores (GPU recommended for training)
  • RAM: 16GB+ (32GB+ recommended for larger models)
  • Storage: 10GB+ (more for storing multiple models)
  • GPU: Optional but highly recommended for training and inference with larger models

Installation Guide

Prerequisites

  • Python 3.6 or later
  • pip package manager
  • Virtual environment (recommended)

Manual Installation

  1. Create and activate a virtual environment (recommended):

    python -m venv transformers-env
source transformers-env/bin/activate  # On Windows: transformers-env\Scripts\activate

  2. Install Transformers with PyTorch:

    pip install transformers[torch]

    Or with TensorFlow:

    pip install transformers[tf]

    For all features:

    pip install transformers[all]

  3. Verify the installation:

    python -c "from transformers import pipeline; print(pipeline('sentiment-analysis')('I love using Hugging Face Transformers!'))"

Note: For detailed installation instructions and GPU support, please refer to the official Hugging Face documentation.

Practical Exercise: Getting Started with Transformers

Now that you have Transformers installed, let’s walk through a simple exercise to help you get familiar with using pre-trained models for NLP tasks.

Step 1: Basic Text Classification

Let’s start with a simple sentiment analysis task:

from transformers import pipeline

# Initialize a sentiment analysis pipeline
classifier = pipeline('sentiment-analysis')

# Analyze some text
results = classifier([
    'I love working with transformers!',
    'This library is not very good.',
    'The API is simple and intuitive.'
])

for result in results:
    print(f"Text: {result['label']}, Score: {result['score']:.4f}")

Step 2: Named Entity Recognition

Now let’s try named entity recognition:

from transformers import pipeline

# Initialize a named entity recognition pipeline
ner = pipeline('ner')

# Analyze some text
text = "Hugging Face was founded in 2016 by Clément Delangue and Julien Chaumond in New York City."
results = ner(text)

# Group entities
entities = {}
for result in results:
    if result['entity'].startswith('B-'):
        entity_type = result['entity'][2:]
        if entity_type not in entities:
            entities[entity_type] = []
        entities[entity_type].append(result['word'])

print("Entities found:")
for entity_type, words in entities.items():
    print(f"{entity_type}: {', '.join(words)}")

Step 3: Text Generation

Let’s try generating text with a pre-trained model:

from transformers import pipeline

# Initialize a text generation pipeline
generator = pipeline('text-generation', model='gpt2')

# Generate text
prompt = "Artificial intelligence is"
results = generator(prompt, max_length=50, num_return_sequences=3)

print("Generated text:")
for i, result in enumerate(results):
    print(f"{i+1}. {result['generated_text']}")

Step 4: Fine-tuning a Model (Advanced)

For more advanced users, here’s how to fine-tune a pre-trained model on your own dataset:

from transformers import AutoModelForSequenceClassification, AutoTokenizer, Trainer, TrainingArguments
from datasets import load_dataset

# Load a pre-trained model and tokenizer
model_name = "distilbert-base-uncased"
model = AutoModelForSequenceClassification.from_pretrained(model_name, num_labels=2)
tokenizer = AutoTokenizer.from_pretrained(model_name)

# Load and preprocess a dataset
dataset = load_dataset("imdb")

def tokenize_function(examples):
    return tokenizer(examples["text"], padding="max_length", truncation=True)

tokenized_datasets = dataset.map(tokenize_function, batched=True)

# Define training arguments
training_args = TrainingArguments(
    output_dir="./results",
    learning_rate=2e-5,
    per_device_train_batch_size=16,
    per_device_eval_batch_size=16,
    num_train_epochs=3,
    weight_decay=0.01,
)

# Create a Trainer
trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=tokenized_datasets["train"],
    eval_dataset=tokenized_datasets["test"],
)

# Fine-tune the model
trainer.train()

# Save the fine-tuned model
model.save_pretrained("./my-fine-tuned-model")
tokenizer.save_pretrained("./my-fine-tuned-model")

Resources