[MINDEXER-142] Add proper documentation with real life examples

**[Martin Todorov](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=carlspring)** opened **[MINDEXER-142](https://issues.apache.org/jira/browse/MINDEXER-142?redirect=false)** and commented

**Task Description**

It would be great to have some proper documentation and not a few cryptic lines and some Javadocs.

The current official documentation is published [here](https://maven.apache.org/maven-indexer/) and it contains very little useful information. Someone who first encounters the project will need to do a lot of digging around in order to extract the useful information they need (mostly by poking around the test code, running through a debugger and hoping to find what they are looking for).

This project has few and very rare contributions (sometimes as bursts of pull requests which take forever to be reviewed and merged), mainly because people feel intimidated to try and make improvements on it, hence the very slow pace of development and low activity.

In my opinion, if it gets some better documentation and, if it's a requirement to continuously improve the documentation (could be done as a check list item in a github issue/pull request template), the project will have a better development pace, as more people will be able to understand it well enough to make actual changes.

[Tamas Cservenak](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=cstamas): Would this be something you could look into as part of the upcoming `6.1.0` release?

**Task List**

Some ideas of things to cover:
* (off) Define key concepts.
* (off) Explain how the indexer works.
* (off) Describe how the index downloading/updating works and what the acceptable frequencies of downloading are.
* (off) Add code examples (and expand on this with actual explanations).
* (off) Add links to the example code, as the current documentation doesn't mention that there is such.
* (off) Cover any other important topics that come to mind and might be useful.

---
No further details from [MINDEXER-142](https://issues.apache.org/jira/browse/MINDEXER-142?redirect=false)

Start with the official documentation at https://maven.apache.org/maven-indexer/ and compare it with the project's test code and existing example code. Define the key concepts, explain indexing and index downloading or updating, document acceptable download frequencies, and add explained code examples with links. Done means these topics are covered clearly enough that a new contributor can understand the project without debugger-driven exploration.

Written by the indexing model from the issue text.