Text Analysis Tutorial: Sequence Labeling & POS/Chunking: An Intermediate, End-to-End Analysis Tutorial

---

# Title & Overview

**Template:** *Sequence Labeling & POS/Chunking: An Intermediate, End-to-End Analysis Tutorial*
**Overview (≤2 sentences):** Learners will implement part-of-speech tagging and chunking pipelines, comparing classical sequence models (CRFs, Perceptrons) with transformer-based token classification. It is intermediate because it emphasizes structured prediction, token-label alignment, and slice-based analysis of sequence labeling errors.

# Purpose

To teach learners how to build and evaluate **sequence labeling models** beyond NER, including POS tagging and phrase chunking. This highlights structured sequence dependencies, error slicing, and reproducibility across classical and neural baselines.

# Prerequisites

* Skills: Python, Git, pandas, ML basics.
* NLP: tokenization, embeddings, BIO tagging, evaluation metrics (token-level accuracy, seqeval F1).
* Tooling: pandas, scikit-learn, Hugging Face Transformers, seqeval/torchmetrics, MLflow, FastAPI.

# Setup Instructions

* Environment: Conda/Poetry (Python 3.11), deterministic seeds.
* Install: pandas, scikit-learn, Hugging Face Transformers + Datasets, seqeval, MLflow, FastAPI.
* Datasets:

* **Small:** Universal Dependencies (English-EWT) for POS tagging.
* **Medium:** CoNLL-2000 for chunking (NP/VP/PP phrases).
* Repo layout:

```
tutorials/t5-sequence-labeling/
├─ notebooks/
├─ src/
│ ├─ data_utils.py
│ ├─ models.py
│ ├─ train.py
│ └─ config.yaml
├─ data/README.md
├─ reports/
└─ tests/
```

# Core Concepts

* **Sequence labeling tasks:** POS tagging (syntactic roles) and chunking (phrase boundaries).
* **Structured prediction:** CRF vs Perceptron sequence models; label dependencies.
* **Byte-level BPE with transformers:** alignment of subwords to labels.
* **Evaluation granularity:** token accuracy vs span-level F1.
* **Error slicing:** performance by tag class, sentence length, domain shift.

# Step-by-Step Walkthrough

1. **Data intake & splits:** load UD (POS) and CoNLL-2000 (chunking); reproducible splits.
2. **Classical baselines:**

* CRF sequence model (e.g., sklearn-crfsuite).
* Averaged Perceptron tagger (spaCy baseline).
3. **Transformer baseline:** DistilBERT (byte-level BPE) with token classification head; fine-tuned on POS/chunking labels.
4. **Evaluation:** token accuracy, entity/chunk F1, per-label F1 via seqeval.
5. **Error analysis:** per-tag confusion (NOUN vs PROPN, NP vs VP), performance on short vs long sentences.
6. **Reporting:** metrics tables, confusion matrices, error examples to `reports/t5-sequence-labeling.md`.
7. *(Optional)* Serve: FastAPI endpoint for POS + chunking predictions with schema validation.

# Hands-On Exercises

* Ablations: CRF vs Perceptron vs Transformer; different context windows; max seq-len variations.
* Robustness: evaluate domain-shift (formal vs informal text).
* Slice analysis: per-tag F1 (rare tags like INTJ, less common phrases).
* Stretch: visualize embeddings for POS-tagged tokens with t-SNE/UMAP.

# Common Pitfalls & Troubleshooting

* **Token–label mismatch:** subword splits complicate sequence alignment.
* **Metrics misuse:** token-level accuracy inflates performance vs F1.
* **Over-cleaning:** lowercasing or lemmatizing harms POS tagging accuracy.
* **OOM:** long sentences in chunking tasks; mitigate with truncation or windowing.
* **CRF tuning:** too many features → memory blowup.

# Best Practices

* Always log config, tokenizer artifacts, and dataset version with MLflow.
* Track per-tag metrics, not just overall accuracy.
* Keep Baseline → Structured Model → Transformer progression clear.
* Unit tests: deterministic token-label alignment, CRF feature consistency.
* Guardrails in serving: max length, schema validation.

# Reflection & Discussion Prompts

* Why do CRFs handle structured dependencies better than Perceptrons?
* How do transformers compare on rare tags vs frequent ones?
* What does POS accuracy gain us in downstream civic NLP tasks?

# Next Steps / Advanced Extensions

* Multilingual POS tagging with XLM-R.
* Semi-supervised sequence labeling (self-training on unlabeled data).
* Integrate POS + chunking features into downstream tasks (NER, parsing).
* Lightweight monitoring of tag distributions in serving.

# Glossary / Key Terms

POS tagging, chunking, CRF, Perceptron, byte-level BPE, token–label alignment, seqeval.

# Additional Resources

* [[Universal Dependencies](https://universaldependencies.org/)](https://universaldependencies.org/)
* [[CoNLL-2000 dataset](https://www.clips.uantwerpen.be/conll2000/chunking/)](https://www.clips.uantwerpen.be/conll2000/chunking/)
* [[spaCy](https://spacy.io/)](https://spacy.io/)
* [[Hugging Face Transformers](https://huggingface.co/docs/transformers)](https://huggingface.co/docs/transformers)
* [[seqeval](https://github.com/chakki-works/seqeval)](https://github.com/chakki-works/seqeval)
* [[MLflow](https://mlflow.org/)](https://mlflow.org/)

# Contributors

Author(s): TBD
Reviewer(s): TBD
Maintainer(s): TBD
Date updated: 2025-09-20
Dataset licenses: UD (varies by treebank, mostly CC BY-SA), CoNLL-2000 (LDC).

# Issues Referenced

Epic: HfLA Text Analysis Tutorials (T0–T14).
This sub-issue: **T5: Sequence Labeling & POS/Chunking**.

---