OpenLake / OpenLake/Student_Database_COSA

[FOSSOVERFLOW-25] feat: Automatic Scraping & Parsing of Academic Calendar

Open
#199 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Contributors needed enhancement Foss OverFlow hard help wanted
Dominant language
JavaScript
Stars
13
Forks
58
Avg merge
20h 27m
Merged PRs (30d)
3

Description

Feature Request: Automatic Scraping & Parsing of Academic Calendar

Problem Statement

Current Issue:
The academic calendar is uploaded on the IIT Bhilai website as a PDF/HTML table, and currently admins must manually copy dates into our Calendar component. This is time-consuming, error-prone, and requires updates every semester/year.


Proposed Solution

✅ Automatic Academic Calendar Scraping
✅ Integration with Existing Calendar UI
  • New events should automatically appear in the existing Calendar component with category-appropriate icons and color coding.
  • Admins should have ability to review/edit scraped data before publishing (optional improvement).
Technical Notes
Layer Requirement
Scraper A cron-triggered script
Data Parsing Match patterns like: "holiday", "exam", "commencement", "registration"
Database Add relevant fields to calendar events table
Retry & Fail-Safe If scraping fails, continue using last known data
Optional Cache PDF locally with versioning for historical comparison

Possible Libraries

  • BeautifulSoup4 / lxml (for HTML scraping)
  • PyMuPDF or pdfplumber if format changes back to PDF

Alternatives Considered

  1. Manual Upload of Events
    ❌ Still requires regular admin effort
  2. Direct API Feed from Institute Website
    ❌ No such API currently exists

Mockups & Visual Examples

  • Checkout the figma design mockups in README

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading the README's Figma mockups and locating the existing Calendar component and calendar events table. Define the scraping, parsing, categorization, scheduling, and failure behavior from the issue, with new events appearing in the UI and the last known data retained when scraping fails.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, mongodb, node.js, react
Domain
backend, database, frontend, full-stack
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.