Analyze schema from messages in a topic

Usecase

The Kafka topics I work with contain JSON encoded messages which follow a certain schema. The possible properties and the value types aren't clear at first glance. However knowing the schema is key to consume a topic and in the end to work with the data. To make this easier Kowl can help us by generating a schema in a best effort fashion from a set of sample messages.

Kowl can fetch the set of samples messages by simply consuming a represantative set of messages.

Challenges / To be clarified:

• Some topics do not follow a strict schema or single messages may be in conflict with the general schema
• Schema changes in the course of time are common
• What is a representative set of sample messages? (number of messages, what offsets etc)
• What encodings do we want to support and what do we show for unsupported encodings? (JSON, XML, ...)
• Generating the schema from the set of messages (do libraries exist for that?)
• In what format do we want to return the recognized schema?

Hints:

This issue complements https://github.com/cloudhut/kowl/issues/49 very well and combining the information from these two solutions should be considered for the best possible user experience.

The issue names no files, tests, or entry points. Start by resolving the listed questions about sample selection, encodings, schema conflicts, and output format; done criteria are not defined yet.