ACM-VIT/scrag

View on GitHub

A flexible web scraper that intelligently adapts to different website structures using multiple extraction strategies (newspaper3k, readability-lxml, BeautifulSoup, and optional headless rendering). It outputs clean, structured data for RAG pipelines or local LLMs, with an optional extension to automatically build RAG indexes from web queries.

Stars: 4
Forks: 15
Open beginner issues: 0
Indexed issues: 4

Dominant language: Python
License: MIT
Last GitHub push: Nov 12, 2025
Latest indexed: Sep 14, 2026
Contributing guide: Contributing guide
Code of conduct: Code of conduct
Beginner labels: hacktoberfest good first issue help wanted
PR merge metrics: No merged PRs in 30d

Open All

4 open issues indexed

feat(frontend): setup UI that will serve as the frontend for the app Open

enhancement good first issue hacktoberfest help wanted

ACM-VIT/scrag#12 · 6 comments · 0 reactions · 1 assignee · Indexed Sep 14, 2026
Optional Selenium/Playwright extractor for JS-heavy pages Open

enhancement hacktoberfest

ACM-VIT/scrag#25 · 2 comments · 0 reactions · 1 assignee · Indexed Sep 14, 2026
Caching and ETag/If-Modified-Since support Open

enhancement good first issue hacktoberfest help wanted

ACM-VIT/scrag#28 · 3 comments · 0 reactions · 2 assignees · Indexed Sep 14, 2026
Feature Request: Multi-URL Scraping with Domain Grouping & RAG-Ready Export Open

ACM-VIT/scrag#39 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 14, 2026

ACM-VIT/scrag

Get new issues in your inbox