Process streams

At the start of the project there was the idea to process Twitter posts. At the moment we are only able to process files. The application should be able to process different streams (e.g. RSS or Twitter posts). So we could continuously fill the database.

The issue names no files, tests, or entry points; start by tracing how the application currently processes files and how processed data reaches the database. Define the stream sources to support, including RSS and Twitter posts, and consider the continuous-ingestion behavior required before deciding what done means.

Written by the indexing model from the issue text.