create class/es that allows to scrape documents using open office
- Dominant language
- Python
- Stars
- 3
- Forks
- 1
- PR merge metrics
- No merged PRs in 30d
Description
Today in Open Knesset the scraping of documents is done differently for each scraping task
for example -
rtf documents are scraped using Rtf15Reader
doc documents are scraped using antiword - which converts the doc to xml which is then parsed
I want all the scrapers to work in a similar way, a possible solution is using Libre Office
see here for a description of using Libre Office to convert documents:
http://ask.libreoffice.org/en/question/2641/convert-to-command-line-parameter/
we can use that command to convert all documents to odf format (which is an xml)
and then from that format scrape the required data (using BeautifulSoup or something similar)
sample use cases for this process:
- parse plenum protocols -
relevant code: https://github.com/hasadna/Open-Knesset/blob/master/plenum/management/commands/parse_plenum_protocols_subcommands/download.py
(ignore the html parsing parts - concentrate only on the parsing of the xml received from antiword)
- get committee meeting protocol -
relevant code: https://github.com/hasadna/Open-Knesset/blob/master/simple/management/commands/syncdata.py
get_committee_protocol_text method
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.