oidlabs-com / oidlabs-com/Lexoid
Apply a file size cap when recursively parsing webpages
Open
Nobody has claimed this yet.
dev
- Dominant language
- Python
- Stars
- 108
- Forks
- 14
- PR merge metrics
- No merged PRs in 30d
Description
The cap can be on the file size or some other attributes. For example, we can define a limit on the number of rows if there's a link to a csv/xlsx file, or a limit on page count if there is a link to a large PDF. We will need to assess what might be a good limit.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No files, tests, or entry points are named. Start by locating the recursive webpage parser and tracing how linked CSV/XLSX files and PDFs are handled; define appropriate size, row, or page limits and document the chosen limits and expected behavior as done criteria.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, web-dev
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100