[DEP] DBoM V2 Architecture

### Enhancement Description

#### Motivation and Scope

Over the last two years, we have gained a deep understanding of the applicability of DBoM Channel-based attestation sharing. We have also invalidated some initial assumptions.

These are the primary pain points that we want to address with this re-architecture:

- DBoM Channel establishment is clunky and requires direct connections to repositories
- The current channel access control mechanism is based on using a third-party trust anchor and communication mechanism (Matrix). We leverage Matrix rooms and private messages to allow DBoM nodes to generate and receive both the secrets and instructions to directly connect to the repository. This is relatively insecure and hard to standardize across repositories due to:
- Direct access to the repository port is provided to the remote node (this is dangerous from a security perspective as it is not
recommended exposing a database endpoint to the outside world with just database authentication
- Revocation and rotation of repository credentials require a unique implementation for each repository. Several implementations lead to more lines of code that we need to maintain and audit for security purposes.
- Any vulnerability that exists in the repository will be exploitable by the remote node. Since we do not intercept the connection between the remote node and the repository, we cannot really apply any mitigation measures.
- Use of Matrix as a handshaking protocol requires the setup of several heavy Matrix services such as the home server, identity server, and more. Matrix (and other alternatives) are full-blown chat servers, with tons of functionality that we will never need to use. DBoM, as a security product, must strive to reduce its attack surface by only implementing the core that is necessary for functionality.
- There is no way to enable Read Audit or implement complex policy rules
- Currently, DBoM implements policies by configuring the RBAC capabilities of the underlying datastore. This means that complex policy rules such as requiring read audit, permission-per-read, and time-limited access to channels are not possible.
- Subscribers cannot truly "subscribe" to a channel in a pub-sub manner. A remote DBoM node cannot scale-ably listen to updates (new/updated) assets on a channel without directly utilizing the change stream functionality presented by the underlying repository. This, again, is difficult to audit as we are limited by the capabilities presented by the repository itself. This also means that the remote DBoM Node needs to care about the type of repository initialized.
- The notarization and storage capabilities of the DBoM node are tightly coupled. The reason we have repository agents for distributed ledgers and transparency logs is exclusively to take advantage of their notarization capabilities. However, most distributed ledgers & transparency logs (with a few exceptions) do not have great query capabilities or performance.

We have had to work around these extensively for both IOTA-Agent and Trillian Agent by either maintaining local state or working around the storage APIs. We also do not have rich query capabilities for any of these agents.

In retrospect, there is no need to couple the notary and storage. You could choose to store your assets on MongoDB, Postgres, or any other compatible database and still choose to notarize them on platforms like Hyperledger Fabric, Ethereum, IOTA, etc. This also means we will get excellent query and commit performance (as databases are built for this) while also having the capability to asynchronously notarize and even have different notaries per channel. This also means that you can only notarize for a subset of critical channels (for instance, why would you want to notarize your dev channel if notarization comes at a cost).

This document describes a high-level summary of changes coupled with a high-level architecture required to achieve these changes. It does not go into implementation specifics such as the API schema.

#### Summary Of Changes

1. **New federation-based channel management model**
All requests to channels from other organizations will now go through their DBoM Node to enable the new policy engine and completely abstract the repository used by the other organization. This is explained in depth in the "Channel Establishment" section of the architecture document.

3. **Separation of the data repository and the notary services**
The data repository will now be independent of the choice of the notary. This means that the notarized attestation of data that is stored in MongoDB, PostgreSQL, or any other database can now reside separately on a distributed ledger, transparency log, or any other notary.

4. **DBoM Nodes will have universally resolvable identities**
DBoM nodes will have a way to establish their identity, by utilizing a multi-factor authentication mechanism that can involve mTLS PKI-based Cryptography and Decentralized Identities, described in the "DBoM Node Identities and URIs for channels" section.

5. **DBoM Nodes will support complex policies and read audits**
With the new federation-based channel management model, all requests to a channel will go through the DBoM node and they no longer hit the repository directly. This means that we can now have complex policies (OpenPolicyAgent-Rego-like) and capabilities like read-audit.

6. **Introduction of a pub-sub service**
In this iteration, remote DBoM nodes (and applications leveraging them) can subscribe to changes on a channel in real time with a repository-agnostic streaming API.

7. **A new iteration of the asset envelope**
The asset envelope is the schema that needs to be followed to commit an asset to a DBoM channel. There is an overhauled schema that is use-case independent. See more in Asset Envelope V2

9. **Complete removal of the RepoID concept at a DBoM Node level**
With the new channel management model, it makes the most sense for one DBoM node to have precisely one local repository configured at a given time. To be clear, the node will still support a variety of local repositories, but will only be associated with one at any given time. This will remove complexities arising from cross-repository operations such as aggregations.

---

- Discussion Link: This Issue
- Architecture/Reference documentation links:
- [High-level Architecture Document](https://docs.google.com/document/d/17LGtrAJYtRtwGuZqPdMcryPVXlZ5sULUKltOHuY9qwM/edit?usp=sharing)
- Primary contact (assignee): @DBOMproject/unisys-technical