Cache the titles, descriptions and subtitles
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 21
- Forks
- 17
- Avg merge
- 5m
- Merged PRs (30d)
- 4
Description
Videos titles, descriptions and subtitles are not yet cached on S3.
They are however not expected to change much and are rather time-consuming to fetch (especially when the video has been translated into 10s of languages)
Titles and descriptions requires to fetch the HTML page of the video for every language and parse it with Bettersoup to extract this.
Subtitles have to be converted to proper format.
We should cache them and only refresh them when someone complains or one in a while, especially if we continue to want to update the ZIM on a very regular basis to fetch the few new videos that have been published.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No file or test is named. Start by tracing the Python scraper paths that fetch TED HTML, parse titles and descriptions, convert subtitles, and upload data to S3. Done means these assets are cached and are refreshed only under the agreed complaint or periodic-refresh conditions.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, python
- Domain
- backend, cloud
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100