Project: Build a Unified Knowledge Layer (Notes, Glossary, & SEO) for Zoomcamp Courses

📝 Background & Objective

Our various Zoomcamp courses (Machine Learning, Data Engineering, MLOps, LLM Zoomcamp, and AI Dev Tools) contain hundreds of hours of highly technical video content.

Currently, navigating this material requires watching or re-watching videos. We want to build a full knowledge layer on top of all Zoomcamp courses. The goal is to generate structured, high-quality written notes for every video, create a shared vocabulary of key concepts, and introduce an automated internal linking system.

By combining written notes, a global glossary, cross-linking, and structured SEO data, we will drastically improve course navigation, the student learning experience, and our long-term search engine visibility.

✅ Core Requirements & Implementation Steps

This is a large-scale project that touches data pipeline automation, Markdown generation, and SEO architecture. You can contribute to the entire system or focus on a specific module below.

1. Automated Notes Generation Pipeline

We need an automated process that turns raw video content into structured written drafts.

• Extraction: Fetch automated transcripts directly from the YouTube videos across our various Zoomcamp playlists.
• Processing & LLM Generation: Preprocess the transcripts and use an LLM to produce structured notes.
• CI/CD Integration: Eventually, this pipeline should be packaged as a GitHub Action that can automatically run and refresh notes whenever courses update or new videos are published. (Note: Automation generates initial drafts; contributors will later manually refine them).

2. Unified Notes Repository & Structure

The output of the automated pipeline must be cleanly organized into a dedicated repository structure.

• Folder Hierarchy: Organized logically by Course → Module → Video.
• Markdown Formatting: Every generated note must follow a unified Markdown template that includes:
  • A clear summary of the video.
  • Key concepts and detailed explanations.
  • Links to related notes and global vocabulary entries.

3. Internal Linking System (Knowledge Graph)

To make navigation seamless, the system must automatically connect related concepts across different modules and courses.

• Detection: Automatically detect related notes using concept extraction, keyword matching, or text embeddings.
• Output: Automatically append a “Related Notes” section to the bottom of each generated Markdown file. This creates a lightweight knowledge graph inside the repository, helping learners and search engines understand how lessons relate to one another.

4. SEO & Structured Data Architecture

To maximize discoverability, the generated notes must be built for modern search engines.

• JSON-LD: Inject JSON-LD structured data into every notes page and vocabulary term. This specifically helps Google interpret the pages as educational resources, definitions, or course materials.
• Metadata: Programmatically generate clear metadata (SEO titles, meta-descriptions, alt text).
• Sitemap: Ensure the pipeline updates or generates a sitemap for the new knowledge layer.

Sources

The automated pipeline will extract lesson videos directly from the modules linked within each course's respective GitHub repository.

This project covers the following Zoomcamps:

The issue names no existing files, tests, or entry points. Start by reviewing the site repository and the linked course repositories to map current content and automation; done would require a defined scope plus the requested notes, glossary links, SEO metadata, JSON-LD, and sitemap workflow.