Need testing framework

Good code has test, and this repo has little testing. I (someone?) should add a comprehensive testing framework. There are some complications here related to the fact that you might want to test against third party websites that change. A good approach might be a (1) pytests that call the API like a client and (2) a webserver that serves a bunch of different content - PDFs, other kinds of files, weird mimetypes, complicated javascript, resource intensive pages, etc. So the tests would ping the scraper, which in turn would scrape localhost:5678, say, which would be our testing playground.

Start by reviewing the proposed pytest API-client approach and the localhost:5678 testing playground described in the issue. Define repeatable coverage for PDFs, other files, unusual MIME types, complicated JavaScript, and resource-intensive pages; done means the scraper is exercised against these cases.

Written by the indexing model from the issue text.