Text Analysis Tutorial: Named Entity Recognition (NER)

---

# Title & Overview

**Template:** *Named Entity Recognition (NER): An Intermediate, End-to-End Analysis Tutorial*
**Overview (≤2 sentences):** Learners will implement NER pipelines using both rule-based/spaCy approaches and transformer-based token classification models. It is intermediate because it requires handling token–label alignment, reproducibility, and error slicing across entity types.

# Purpose

To teach learners how to build, evaluate, and compare **defensible NER baselines** versus transformer models, and how to interpret performance differences by entity type, span length, and domain. This emphasizes error analysis and reproducibility, not just getting entities out of text.

# Prerequisites

* Skills: Python, Git, pandas, ML basics.
* NLP: tokenization, embeddings, BIO tagging, evaluation metrics (F1, precision, recall).
* Tooling: pandas, scikit-learn, spaCy, Hugging Face Transformers, seqeval/torchmetrics, MLflow, FastAPI.

# Setup Instructions

* Environment: Conda/Poetry (Python 3.11), deterministic seeds.
* Install: pandas, scikit-learn, spaCy, Hugging Face Transformers + Datasets, seqeval, MLflow, FastAPI.
* Datasets:

* **Small:** CoNLL-2003 (English NER: PER/LOC/ORG/MISC).
* **Medium:** OntoNotes 5 (broad domains, multiple entity types).
* Repo layout:

```
tutorials/t4-ner/
├─ notebooks/
├─ src/
│ ├─ data_utils.py
│ ├─ models.py
│ ├─ train.py
│ └─ config.yaml
├─ data/README.md
├─ reports/
└─ tests/
```

# Core Concepts

* **BIO tagging scheme:** Begin/Inside/Outside labeling of entity spans.
* **Byte-level BPE tokenization:** alignment challenges when subwords split entities.
* **Classical vs neural approaches:** spaCy’s rule-based + statistical vs transformer token-classification (e.g., DistilBERT).
* **Evaluation granularity:** entity-level F1 vs token-level accuracy.
* **Error slicing:** entity-type F1, short vs long entities, capitalization sensitivity.

# Step-by-Step Walkthrough

1. **Data intake & splits:** load CoNLL-2003/Ontonotes; stratified reproducible splits.
2. **Classical baseline:** spaCy NER pipeline (out-of-the-box, fine-tune optional).
3. **Transformer baseline:**

* DistilBERT (byte-level BPE).
* Token classification head (linear layer on top).
* Training with label alignment (map subword pieces to labels).
4. **Evaluation:** entity-level precision/recall/F1 (seqeval); token-level accuracy as sanity.
5. **Error analysis:** per-entity-type F1, confusion examples (ORG vs LOC), span length errors.
6. **Reporting:** metrics tables, error slices, example entities to `reports/t4-ner.md`.
7. *(Optional)* Serve: FastAPI endpoint that returns entity spans and types, validates input, and logs requests.

# Hands-On Exercises

* Ablations: spaCy vs transformer; different max seq-len (128 vs 256); freeze vs full fine-tune.
* Robustness: test casing (UPPERCASE, lowercase), punctuation noise, domain-shift text.
* Slice analysis: compare performance across PER, LOC, ORG, MISC classes.
* Stretch: extend to multilingual NER with XLM-R.

# Common Pitfalls & Troubleshooting

* **Token–label mismatch:** subword splits → alignment errors if not handled.
* **Metrics misuse:** token-level accuracy inflates performance vs entity-level F1.
* **Over-cleaning:** lowercasing can erase entity cues.
* **OOM:** long docs; mitigate with truncation or sliding windows.
* **Silent drift:** tokenizer vocab changes alter entity alignment — must log artifacts.

# Best Practices

* Track entity-type performance, not just overall.
* Log config, seeds, tokenizer vocab/merges, dataset fingerprints with MLflow.
* Unit tests: token-label alignment correctness, deterministic splits.
* Serve with schema guardrails: max length, allowed entity types, input validation.
* Keep Baseline → Transformer narrative clear.

# Reflection & Discussion Prompts

* Why do transformers often outperform spaCy NER on rare/multi-token entities?
* How does normalization (lowercasing, NFKC) affect entity recognition?
* What’s the trade-off between entity recall and precision in real-world applications?

# Next Steps / Advanced Extensions

* Apply domain adaptation (fine-tune on civic-text corpus).
* Experiment with nested entities and overlapping spans.
* Explore active learning: identify hardest sentences for annotation.
* Lightweight deployment: batch inference + drift monitoring.

# Glossary / Key Terms

BIO tagging, entity-level F1, token–label alignment, byte-level BPE, DistilBERT, seqeval, slice analysis.

# Additional Resources

* [[spaCy](https://spacy.io/)](https://spacy.io/)
* [[Hugging Face Transformers](https://huggingface.co/docs/transformers)](https://huggingface.co/docs/transformers)
* [[Hugging Face Datasets](https://huggingface.co/datasets)](https://huggingface.co/datasets)
* [[seqeval](https://github.com/chakki-works/seqeval)](https://github.com/chakki-works/seqeval)
* [[MLflow](https://mlflow.org/)](https://mlflow.org/)
* [[FastAPI](https://fastapi.tiangolo.com/)](https://fastapi.tiangolo.com/)

# Contributors

Author(s): TBD
Reviewer(s): TBD
Maintainer(s): TBD
Date updated: 2025-09-20
Dataset licenses: CoNLL-2003 (LDC license), OntoNotes (LDC license).

# Issues Referenced

Epic: HfLA Text Analysis Tutorials (T0–T14).
This sub-issue: **T4: Named Entity Recognition (NER)**.

---