Hangfire 2.0 roadmap

### 2021 Update

The list below is more like a dream than a roadmap – it contains significant changes no one except new users would like to see. It's a great list of features for the new product, but I don't see how it can be possible to move forward these items while maintaining backward compatibility, the essential feature in Hangfire.

### Programming Model

In short: generalization of a background job, more `Task`-like semantics for background jobs.

* Pending, Completed, Faulted, Canceled states for regular background jobs.
* Result of an antecedent job is available in continuations.
* Continuations can be run synchronously, i.e. on the same worker without any storage call.
* Recurring jobs (and batches, name will be changed) are regular jobs with just a special processing.
* Continuously-running processes are exposed as regular jobs also.
* Support for `Job.WaitAny` and `Job.WaitAll` background job types.
* Queues can be used to isolate different code bases without creating additional storage.
* Internals know nothing about user types, to allow shared monitoring.
* Full async/await programming model support for background jobs via dedicated threads.
* IoC container scope starts much earlier in the processing pipeline.

Goals:

* Even simpler understanding of background jobs due to well-known `Task`-like states.
* Better composition for jobs with support for results, ability to use map-reduce model.
* Simplified continuously-running processes without need for distributed locks.
* Support for microservices architecture by queue-based isolation.
* More performance for recurring jobs without any iterations.

### Storage Model

In short: support for coarse-grained operations, idempotent writes to support eventually-consistent storages.

1. Two general aspects of a storage are message queues and background jobs, other types are removed.
2. Storage transactions are exposed directly to user code to support atomic and batched operations.
3. All storage operations are coarse-grained to support batch processing.
4. Each command in a storage transaction is idempotent to support eventually-consistent storages.
5. Storages that support automatic sharding can use it by implementing custom transaction log.
6. Built-in support for time-based scheduling in message queues for faster delayed jobs.
7. Storage abstractions know nothing about user types, data is serialized before passing to thee storage API.
8. Full support for asynchronous I/O for storages in every query.
9. Binary data and payloads for variable-length fields to reduce payload size.

Goals:

* Better network and storage utilization by using batched operations.
* Get rid of any bottlenecks to support nearly linear scaling for background processing.
* Simplified storage implementations for eventually consistent storages without implementing rollbacks via Two-Phase Commit protocol.
* Better performance for scheduled (time-based) processing.
* Redis Cluster and Cassandra can be utilized with any number of nodes with almost linear processing scale.

Start by reviewing the Programming Model and Storage Model sections, then compare their goals with Hangfire's current backward-compatibility constraints. The issue names no files, tests, entry points, or concrete acceptance criteria, so a contributor would need the maintainers to select and scope a specific roadmap item before implementation can begin.

Written by the indexing model from the issue text.