[Meta] End to End ACKs / Queueless Mode

ROADMAP

- [ ] Add tests to E2E/Other Pipeline code
- [ ] Benchmark feature
- [ ] Create blog post
- [ ] Experimental feature flag in master / 6.x

One desirable property for Logstash would be the ability to operate in as stateless a mode as possible. One hindrance to this is the fact that with the `memory` queue, data is lost if the process crashes. With a `persistent` queue, data is lost if there is a permanent machine failure.

This is unavoidable with some input/output combos, such as raw TCP, where there is no system of acknowledgements. For others, however, like the `beats` (with client side spooling) or `kafka` inputs, or the `elasticsearch` or `kafka` outputs we can provide the same guarantees with no need to keep state on disk.

When using these technologies it would be nice to be able to have a third `queueless` option, that would ACK E2E, and be able to withstand a permanent Logstash machine failure. This would not be able to handle a permanent input client failure of course, since it depends on that for replay.

In this thread let's discuss approaches to implementing this future.

Key questions:

1. How does the user experience change? What do users need to know? Do they need to tune Logstash differently? What is the UX here?
2. Which internals need to change?
3. Do we need to make any changes to the plugin API or any plugins to support these changes?
4. What do we need from an input source to make it queueless?