OpenLake / OpenLake/Student_Database_COSA
[FOSSOVERFLOW-25] feat: Automatic Scraping & Parsing of Academic Calendar
Nobody has claimed this yet.
- Dominant language
- JavaScript
- Stars
- 13
- Forks
- 58
- Avg merge
- 20h 27m
- Merged PRs (30d)
- 3
Description
Feature Request: Automatic Scraping & Parsing of Academic Calendar
Problem Statement
Current Issue:
The academic calendar is uploaded on the IIT Bhilai website as a PDF/HTML table, and currently admins must manually copy dates into our Calendar component. This is time-consuming, error-prone, and requires updates every semester/year.
Proposed Solution
✅ Automatic Academic Calendar Scraping
- Add a backend service that periodically scrapes the academic calendar from:
https://www.iitbhilai.ac.in/index.php?pid=aca_calendar - Parse events and categorize them like "academic", "holiday", etc
✅ Integration with Existing Calendar UI
- New events should automatically appear in the existing Calendar component with category-appropriate icons and color coding.
- Admins should have ability to review/edit scraped data before publishing (optional improvement).
Technical Notes
| Layer | Requirement |
|---|---|
| Scraper | A cron-triggered script |
| Data Parsing | Match patterns like: "holiday", "exam", "commencement", "registration" |
| Database | Add relevant fields to calendar events table |
| Retry & Fail-Safe | If scraping fails, continue using last known data |
| Optional | Cache PDF locally with versioning for historical comparison |
Possible Libraries
BeautifulSoup4/lxml(for HTML scraping)PyMuPDForpdfplumberif format changes back to PDF
Alternatives Considered
- Manual Upload of Events
❌ Still requires regular admin effort - Direct API Feed from Institute Website
❌ No such API currently exists
Mockups & Visual Examples
- Checkout the figma design mockups in README
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the README's Figma mockups and locating the existing Calendar component and calendar events table. Define the scraping, parsing, categorization, scheduling, and failure behavior from the issue, with new events appearing in the UI and the last known data retained when scraping fails.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript, mongodb, node.js, react
- Domain
- backend, database, frontend, full-stack
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100