ACM-VIT / ACM-VIT/scrag

Optional Selenium/Playwright extractor for JS-heavy pages

Open
#25 2 comments 0 reactions 1 assignee Claimed by @samarthnaikk View on GitHub
enhancement hacktoberfest
Dominant language
Python
Stars
4
Forks
15
PR merge metrics
No merged PRs in 30d

Description

## Why
The architecture mentions an extractor for JS-heavy pages, but it hasn't been implemented. Many modern websites require JavaScript execution to properly extract content.

## Tasks
- Introduce an optional dependency path (e.g., extras: web-render).
- Implement an extractor that uses Selenium or Playwright to handle JS-heavy pages with a hard timeout.
- Document the heavy footprint and recommend skipping in CI environments.

## Acceptance Criteria
- There is a config key (e.g., selenium) that resolves when the extra dependency is installed and fails cleanly when it is not.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.