internetarchive / internetarchive/openlibrary-bots
Large ONIX Files require iterparse (else out-of-memory errors)
Open
ONIX bot
requirement
- Dominant language
- Python
- Stars
- 81
- Forks
- 67
- PR merge metrics
- No merged PRs in 30d
Description
lxml has an option for iteratively parsing xml files which we will want to implement, otherwise, we will run out of memory. Here's an example:
https://lxml.de/parsing.html#iterparse-and-iterwalk
Contributor guide
Assessment
This issue has not been assessed yet.