Gateway Specification

Document

This is a very rough specification of how gateways will work in Joystream, in particular the APIs, which hopefully should be conceptually clear enough to warrant implementation. It should be noted that this is a specification only because of its relatively high level of specificity, this should however not be mistaken to imply that these are immutable design and implementation decisions. This should should be taken as a concrete starting point for an overall approach, where many details may need to change.

Background

1. Advertising is indispensable near term: Reaching a broad audience of content consumers in the near term, in a way which is economically sustainable, means that the system must have ways of capturing value that does not depend on consumers having a wallet and tokens. This value capture is required to cover the cost of the computing infrastructure, such as storage and queries, but also to entice content creators to publish and create content on the system. The only mechanism that satisfies this goal is some kind of advertising system.
2. Advertising requires Sybil prevention: The most important technical and policy requirement of any advertising system is the ability to correctly distinguish genuine from fraudulent impressions.
3. Sybil prevention depends on app delivery: Sybil prevention is only really practical when the platform controls the delivery of the user facing experience. Such control enables detection and prevention of abusive user practices, for example by using techniques such as Captchas or email registration. As an example of the most blatant attack possible, anyone running a Joystream application could simply replace all the advertising endogenous to the Joystream system with their own integrations. A milder attack is just normal client side ad-blocking in browsers.
4. Blockchains are isolated from web 2.0 app delivery: Asserting control over web 2.0 assets for distributing apps, such as

• app store entries (phones, TVs, etc.)
• ICANN domain names
• desktop app certificates
  is a major barrier for Blockchain systems. Alternative systems are being deployed, their adoption is likely to take a long time.

1. Conclusion: Delivery of apps must be a role for a sustainable way for Joystream to reach a broad consumer audience in a sustainable way in the near term.

Idea

The idea for how to solve problem described is to couple advertising and app-delivery at the same layer, called a gateway, exogenous to the core Joystream protocol. This layer sits on top of the core services provided by the protocol, including content creation, and pays for utilisation of these services on behalf of its screened users. This incentivises gateway operator to properly internalise the cost of Sybils, and thus invest in screening them properly, because the gateway has to pay for the abusive users. The gateway is in the best position to make this investment because it has more information, in particular because it own the app ui, and can cross-reference self-reported self-reported resource utilisation from users with impression and fraud information from advertising system and service providers.

Proposal

Formalise the operation of a Joystream user facing application, called a gateway, into a proper role, called a gateway operator. Each gateway has its own set of users, called gateway users, and manages the relationship with them through it's own discretionary Sybil control and monetisation system independent of Joystream, and it also is responsible for within-protocol payment for the service usage of it's users. In what follows the different components of this architecture, and their interactions, are described at a high level.

Architecture

The following figure gives high level overview of the architecture of for how gateways interact with the rest of the system.

The ❤️ of the protocol

Strapped for time? then only read this section.

At the heart of this protocol is a strategic interaction between the gateway and the service provider. The gateway is attempting to ensure continued service for its users, and the service provider is attempting to extract payment commitment from the gateway that reflects the service provided. The gateway is supposed to measure how much service is being rendered by accepting service consumption reports from its gateway users, which it also undertakes to ensure are honest in its own interest, using its own policies. The service providers can directly measure the rendered service by provisioning it to users credentialed as legitimate by the gateway. On this basis, the service provider will periodically attempt to obtain an off-chain attestation from the gateway to this effect. The gateway will compare the request to its own internal long and short run accounts of rendered service, and sign off on requests that are within acceptable bounds. The idea is that the repeated interaction should induce both parties to collaborate honestly in order to unlock gain from trade on future payment/service rounds. Accumulating discrepancies between requested payment magnitudes from the provider, and the reports accumulated by the gateway, can be dealt with outside the core gateway-provider protocol, and with the help of the gateway lead.

The following figure attempts to summarise the interactions.

Concepts

Service Type

At any given time, the set of different services on offer by service providers could be partitioned into families of Service Types.

Unit Of Service

For a given service type, there will be some unit which describes some base quantity of service that can be rendered, accounted and paid for, to gateway users. The token price for such service will be denominated in this unit. Example could for example be some number of bytes, in the context of bandwidth services, or it could be queries, in the context of query node service. It could also be something abstract, for example to reflect the fact that different service requests could have qualitatively distinctions, such as the fact that the bytes serviced from some piece of content could be weighted differently than bytes from another.

Service Channel

A Service Channel represents an on-chain payment channel between a given gateway and service provider. There is at most a single such channel between each such pair at any given time.

Briefly explained, such a channel, which is a one-to-one unidirectional channel, requires that the payor (gateway) locks up an amount of funds in advance with the payee (service provider) that is sufficient to cover total required spending over some period of time into the future. The payor then signs statements, called payments, which attests to the claim that the payee has provided a given amount of total service over the given channel. These signatures are provided off-chain regularly over time, corresponding to increasing total quantities of service rendered, hence the last such statement is always the most valuable to the payee. The payee can at any time settle such a payment with the chain, resulting in the funds being released in a split between the payee, coins burned for the platform, and refund for the payor. The terms of this split is determined by service unit prices committed to when the channel was created. Obviously using the last signed payment is desirable for the payee, as it is going to be the largest claim, and the payee must never accept any signature which exceeds the total amount of locked capital at any given time as valid. Such a settlement can only be done once. Lastly, the payor can unilaterally request a refund at any time, in case the payee stops cooperating, which would return the full amount to the payor. To avoid a hostile race condition, this refund has a refund period during which the payee could observe a possibly malicious refund attempt, and publish the last signed payment in their favour in advance. It also possible for the payor to increase the amount of locked funds at any time, or for the payee to release locked funds in favour of the payor at any time, both of which are intended to facilitate cooperative capital management.

One could easily wonder why such an off-chain solution is required? Why not just pay on-chain? The problem is the number of service providers likely to be involved, it could easily be in the hundreds, and each has a separate relationship with some subset of gateways, of which there are certainly much fewer, but still likely more than a single one. Combine this with the desire to make the interaction low risk, by having a high settlement frequency, then this starts taking up a very large share of block space, possibly becoming entirely infeasible at moderate levels of scale.

Service Providers

A service provider a network level infrastructure provider providing some service to end users. Each provider is controlled by an operator, and this operator will hold the role of a worker in one of the on-chain working groups.

API Sketch

# Depends on common.grapqhl in same folder

# Service provider submitted payment request to gateway with some possible outcome
type PaymentAttempt {

  request: PaymentSignatureRequest!

  # When request was sent
  request_timestamp: Int!

  # Host initiating the call
  destination_host: String!

  # Result from host, if provided yet or ever
  result: PaymentSignatureRequestResult

  # Local timestamp for when result was provided, if ever.
  result_timestamp: Int!

}

type ActiveServiceChannel {

  # On-chain id of service channel
  service_channel_id: ID!

  # This is the actual estimated service level of the provider.
  current_estimated_total_service_level: Int!

  # Log of attempted payments
  payment_attempts: [PaymentAttempt!]
}

type ServiceProviderStatus {

  # Number of active users using this provider
  current_number_of_active_users: Int!

  # Number seconds since this provider was started
  node_uptime: Int!

  # Status of all active channels
  active_channels: [ActiveServiceChannel!]

}

type Query {

  # --- for operator ---

  # Missing!
  # Some ideas for the future

  # --- for all ---

  # Id of this gateway
  service_provider_id(): Int!

  # Type of service provider
  type(): ServiceType!

  # Status of service provider
  status(): ServiceProviderStatus!

}

type Mutation {

  # --- for operator ---

  # Missing!
  # Some ideas for the future

  # --- for gateway user ---

  # Resource request left out.
  # Must be mutation due to side-effects in resource accounting.
  # I think the result here should distinguish various error cases, like
  # - you are blocked ()
  # - your gateway has signed for all locked funds, and needs to reup.
  # - your gateway has failed to provid paymetn signature
  # etc.
  # request_service(gateway_auth_token, ... ): ...

  # --- for gateway ---

  # authentication stuff?
}

Gateway

A gateway is a user facing application delivery through traditional web 2.0 infrastructure, such DNS resolvable websites, mobile, tablet and TV app stores, and equivalents. Each gateway has its own bespoke mechanism for gating access to and monetising users. What mechanism may be relevant is not part of this proposal, and will likely vary with time. Each gateway is operated by a gateway operator, which is a new staked role in the system. Gateway operators participate in their own working group, meaning they are supervised by a leader, must be explicitly hired, may earn periodic rewards and operate under risk of slashing sanctions or termination.

API Sketch

# NOTE: This type almost certainly needs to be enriched into a union where variants
# carry type specific parameters that describe the service type with sufficient granularity
# so as to allow client side mapping of specific resource requests to suitable providers.
#
# Example: Bandwidth type could be enriched with geographic information providing latency
# enhancement
enum ServiceType {
  Bandwidth

  # future
  #Queries,
  #Livestreaming
}


# A service provider
type ServiceProvider {

  # Whether user can currently receive service from this provider.
  availability: Boolean!

  # Identifier of service provider
  service_provider_id: Int!

  # Identifier for the active service channel
  current_service_channel_id: Int!

  # Type of service provider
  service_type: ServiceType!

  # Hosts nodes operated by provider
  hosts: [String!]
}

# Constraints on how a given user can access a given service provider.
type UserOnProviderAccessConstraints {

  # Whether user has access currently.
  blocked: Boolean!

  # Hex encoded authorization token.
  gateway_user_auth_token_on_gateway: String!

  # Maximum number of simultaneous requests.
  max_simultaneous_requests: Int!

  # Maximum cumulative service count over the course of a second.
  max_service_count_per_second: Int

  # Maximum cumulative service count per day.
  max_service_count_day: Int

  # Maximum cumulative service count per week.
  max_service_count_per_week: Int
}

type UserSpecificProviderProfile {

  # Service provider.
  service_provider: ServiceProvider!

  # A report must be sent to the gateway after this multiple of
  # successful service requests from this provider.
  send_service_consumption_report_every_successful_request_multiple: Int!

  # Constraints.
  constraints: UserOnProviderAccessConstraints!
}

type GatewayUser {

  # Identifier of user on gateway.
  gateway_user_id: Int!

}

type ProviderSpecificUserProfile {

  # The gateway user
  gateway_user: GatewayUser!

  # Constraints.
  constraints: UserOnProviderAccessConstraints!
}

# A successful result of authentication attempt of user with gateway.
type UserLoggedInOnGatewayResult {

  # Identifier of user on gateway.
  gateway_user_id: Int!

  # Authorization token for interacting with gateway.
  gateway_user_auth_token_on_gateway: String!

  # Profiles for all providers relevant to user.
  user_specific_provider_profiles : [UserSpecificProviderProfile!]
}

# A failed result of authentication attempt of user with gateway.
type FailedUserLoginResult {

  # Error message
  message: String!
}

# Result of user authentication attempt with gateway.
union UserOnGatewayLoginResult = FailedUserLoginResult | UserLoggedInOnGatewayResult

# Successful signing of a payment request from a service provider
type PaymentSigningEvent {

  # Index of payment signing on
  payment_index: Int!

  # Service level signed.
  # This is the service level the provider requested signature for, it does not in principle
  # match service level recorded by gateway.
  new_total_service_level_signed: Int!

  # This is the actual estimated service level of the gateway at the time of this signing.
  estimated_total_service_level: Int!

  # When signature was made
  timestamp: Int!

  # Payment signature
  signature: String!

  # Commitment over service report
  service_provider_service_report_commitment: String!

  # Commitment over service report accumulated by gateway since last Payment
  gateway_service_report_commitment: String!

  # Host initiating the call
  caller_host: String!
}

# An active service channel
type ActiveServiceChannel {

  # On-chain id of service channel
  service_channel_id: ID!

  # This is the actual estimated service level of the gateway.
  current_estimated_total_service_level: Int!

  # All signature events in the channel so far
  payment_signature_events: [PaymentSigningEvent!]
}

# Self-reported status of a gateway
type GatewayStatus {

  # Number of active users on this gateway
  current_number_of_active_users: Int!

  # Number seconds since this gateway was started
  node_uptime: Int!

  # Status of all active channels
  active_channels: [ActiveServiceChannel!]
}

type AuthChallenge {
  nonce: String!
}

type UnrecognizedServiceProvider {
  /* empty variant */
}

union AuthChallengeInititationResult = AuthChallenge | UnrecognizedServiceProvider

type BadAuthResponse {
  /* empty variant */
}

type SuccessfulAuthResponseValidation {

  provider_auth_token: String!

  status: GatewayStatus!

  provider_specific_user_profiles : [ProviderSpecificUserProfile!]
}

union ValidateAuthResponseResult = SuccessfulAuthResponseValidation | BadAuthResponse

type Query {

  # --- for operator ---

  # Missing!
  # Some ideas for the future
  #
  # - List all users and their status
  # - Get a specific user status
  # - List all service providers
  # - Get one specific service provider
  # - List various policy variables

  # --- for users ---

  # Get all provider profiles for gateway user.
  get_user_specific_provider_profiles(gateway_user_auth_token_on_gateway: String!): [UserSpecificProviderProfile!]

  # --- for providers ---

  # Get user profiles for a given provider
  get_provider_specific_user_profiles(service_provider_auth_token_on_gateway: String!): [ProviderSpecificUserProfile!]

  # --- for all ---

  # Id of this gateway
  gateway_id(): Int!

  # How to locate the apps for this gateway
  app_URIs(): [Strings!]

  # Status of this gateway
  status(): GatewayStatus!

}

# A request for service where the service provider returned the expected resource in a timely manner.
# NOTE: a bit rough perhaps, could be modelled more strictly, but this will make it
# depend on service type.
type SuccessfulServiceRequest {

  # Identifier used for request.
  request_id: ID!

  # Host which serviced request.
  service_provider_host: String!

  # Client side timestamp for request submission.
  sent_timestamp: Int!

  # Client side time from request submission to initial response.
  request_duration: Int!

  # Raw request payload.
  raw_requests: [String!]

  # Service weight of request.
  service_count: Int!
}

# A report describing history of successful service received from a given service provider.
type ServiceConsumptionReport {

  # Service report identifier.
  # Should be unique across all providers for the gateway user.
  report_nr: ID!

  # Service provider from whom service was received.
  service_provider_id: Int!

  # List of successful service events from the given provider, across all hosts.
  # Should be ordered chronologically.
  successful_requests: [SuccessfulServiceRequest!]

  # Local timestamp at which report was created.
  compiled_at_timestamp: Int!
}

# Problem varieties that may occur.
# NOTE: It may be worth turning this into a union type in the future to
# decorate cases with payloads with more information
enum ServiceProviderProblemType {

  # It was not possible to open a TCP connection.
  CannotConnectToHost

  # Provider dropped connection prematurely.
  ConnectionDropped

  # Provider explicitly denied serving user.
  RequestDenied

  # There is no response at all after request is sent.
  DoesNotRespond

  # There is a response, but it is not completed in a timely manner.
  DelayedResponse

  # A timely response is made, but the structure or payload  is not valid.
  MalformedResponse
}

# A problem report from a given gateway user.
type ServiceProviderProblemReport {

  # Problem report identifier.
  # Should be unique across all providers for the gateway user.
  problem_id: ID!

  # Service provider with whom the problem has occurred.
  service_provider_id: Int!

  # Representative host of service provider with whom problem has occurred.
  host: String!

  # The variety of problem
  type: ServiceProviderProblemType!
}

type PaymentSignatureRequest {

  # Unique request identifier
  payment_signature_request_id: ID!

  # Service level request.
  new_total_service_level_requested_paid_for: Int!

  # Commitment over service report
  service_provider_service_report_commitment: String!

}

# Sucessful result returned in response to a payment signing request
type PaymentSignatureRequestSuccessful {

  # Unique identifier across all results
  payment_signature_result_id: ID!

  # Payment signature
  signature: String!

  # This is the actual estimated service level of the gateway.
  current_estimated_total_service_level: Int!
}

# Failed result returned in response to a payment signing request
type PaymentSignatureRequestFailure {

  # Unique identifier across all results
  payment_signature_result_id: ID!

  # This is the actual estimated service level of the gateway.
  current_estimated_total_service_level: Int!

  # cause
  error: String!
}

union PaymentSignatureRequestResult = PaymentSignatureRequestSuccessful | PaymentSignatureRequestFailure

type Mutation {

  # --- for operator ---

  # Missing!
  # Some ideas for the future
  #
  # - Update status of a given user
  # - Update status of service provider
  # - Update various policy variables

  # --- for users ---

  # User signs up
  # NOTE: This is here more as an example
  signup_user(user_name: String!, password: String!, token: String!): SignupUserResult!

  # Gateway user login
  login_user(user_name: String!, password: String!): UserOnGatewayLoginResult!

  # Logs out user
  logout_user(gateway_user_auth_token_on_gateway: String!): Boolean!

  # Gateway user reports service consumed from a provider
  report_service_consumed(gateway_user_auth_token_on_gateway: String!, report: ServiceConsumptionReport!)

  # Gateway user reports problem dealing with a provider host
  report_service_provider_problem(gateway_user_auth_token_on_gateway: String!, report: ServiceProviderProblemReport!)

  # --- for providers ---

  # Initiate authentication steps given service provider.
  initiate_auth_challenge(service_provider_id: Int!): AuthChallengeInititationResult!

  # Validation of signature on auth challenge nonce.
  validate_auth_response(signature: String!): ValidateAuthResponseResult!

  # Provider requesting signature for service rendered.
  sign_payment(service_provider_auth_token_on_gateway: String!, request: PaymentSignatureRequest!) : PaymentSignatureRequestResult!

}

# Events for service provider
type ProviderSpecificUserProfileAdded {

  # Current profile
  profile: ProviderSpecificUserProfile!
}

type ProviderSpecificUserProfileUpdated {
  profile: ProviderSpecificUserProfile!
}

type ProviderSpecificUserProfileRemoved {

  # Identifier of user on gateway.
  gateway_user_id: Int!
}

union ServiceProviderEvent = ProviderSpecificUserProfileAdded | ProviderSpecificUserProfileRemoved | ProviderSpecificUserProfileUpdated

# Events for gateway users
type NewUserSpecificProviderProfile {
  profile: UserSpecificProviderProfile!
}

type RemovedUserSpecificProviderProfile {
  service_channel_id: Int!
}

type UpdatedUserSpecificProviderProfile {
  profile: UserSpecificProviderProfile!
}

type GatewaySessionEnded {
  message: String
}

union GatewayUserEvent = NewUserSpecificProviderProfile | RemovedUserSpecificProviderProfile | UpdatedUserSpecificProviderProfile | GatewaySessionEnded

type Subscription {

  # Events for service providers
  serviceProviderEvents(service_provider_auth_token_on_gateway: String!): ServiceProviderEvent!

  # Events for gateway users.
  gatewayUserEvents(gateway_user_auth_token_on_gateway: String!): GatewayUserEvent!

  # Events for operator.
  # missing
}

Gateway User

A gateway user is someone granted the right to use some application powered by the service providers servicing a given gateway.

Gateway Working Group

The gateway subsystem has a working group for organizing the gateways. The service providers are not in this working group, but likely in other type specific working groups.

Gateway Runtime Module

A new dedicate runtime module for coordinating value transfer between gateways, service providers and the platform is required.

Roles

The gateway module has the following roles:

• Gateway Lead: Is the lead in the corresponding working group.
• Gateway Operator: Is a worker in the corresponding working group, and fills the role of operating a gateway service. Is identified by the identifier of the worker in the gateway working group. Once worker has been hired in this group, it can immediately start creating channels with existing service providers.
• Service Provider: Is a worker in one of the working groups that correspond to a sub-system which renders service to gateway users. Is identified by a unique positive integer, and the operator is identified by a service provider worker id, which is a combination of the and identification of the working group and the worker identifier in that working group. It is the lead who creates service providers that can act in the gateway subsystem, and also determines the platform cut in payments to this provider per unit of service being settled, and also the refund period for channels with this provider. The provider can be removed only by the lead.

Authentication and Fallback Accounts

Gateways and service providers authenticate using the role account of the corresponding worker. This creates the problem that they may leave or be terminated from that working group while there are live channels lingering in the gateway module. The way this is resolved is by the introduction of so called fallback accounts. These accounts can be used to wind down a channel, as a supplement to the normal role account, in the event that the worker role has been removed.

Types

/// A type of service.
enum ServiceProviderWorkingGroup {
  Bandwidth,
  // Query,
  // Livestreaming,
  // ...
}

/// Identifier for the working group worker that acts as the operator role for a service provider
struct ServiceProviderWorkerId {

  /// Worker identifier
  worker_id: Id,

  /// Type of service, which identifier
  group: ServiceProviderWorkingGroup
}

/// State of a `ServiceChannel`
enum ServiceChannelState {
  Pending,
  Operational,
  RefundInitiated( /* initiated in block */ BlockNumber),
}

/// Payment channel between a given gateway and service provider
struct ServiceChannel {

  /// Identifier of gateway side of channel
  pub gateway_worker_id: WorkerId,

  /// Fallback account of gateway.
  pub gateway_worker_fallback_account: AccountId,

  /// Service provider end of channel.
  pub service_provider_id: ServiceProviderId,

  /// Fallback account of service provider.
  pub service_provider_fallback_account: AccountId,

  /// Number of tokens
  pub locked_amount: Balance,

  /// Number of blocks from refund is initiated until it is
  pub refund_delay_period: BlockNumber,

  /// Number of tokens burned for platform per unit of service during channel settlement.
  pub platform_price: Balance,

  /// Number of tokens released to provider per unit of service during channel settlement.
  pub provider_price: Balance,

  /// State of channel
  pub state: ServiceChannelState
}

/// A service provider.
struct ServiceProvider {

  /// Identifier of worker.
  service_provider_worker_id: ServiceProviderWorkerId

  /// Platform price per unit of service to provider.
  platform_service_price_for_new_channels: Balance

  /// Refund period for
  refund_period_for_new_channels: BlockNumber
}

State

decl_storage!{

  /// Service Channels map <== remember to throw them out when they are done, i.e. cancelled, settled or refund completed
  /// Next service provider id
  /// Service Providers map
}

Extrinsics

decl_module!{

  // --- for gateways ---

  /// ...
  pub fn create_channel();

  /// ...
  pub fn cancel_channel();

  /// ...
  pub fn initiate_refund_channel();

  /// ...
  pub fn complete_refund_channel();

  /// ...
  pub fn increase_channel_capital();

  /// ...
  pub fn update_gateway_fallback_account();

  // --- for service providers ---

  /// ...
  pub fn confirm_channel();

  /// ...
  pub fn update_service_provider_account();

  /// ...
  pub fn decrease_channel_capital();

  /// ...
  pub fn settle_channel();

  // --- for both ---

  /// ...
  pub fn announce_new_host_set();

  // --- for lead ---

  pub create_service_provider();

  pub update_service_provider_terms();

  pub remove_service_provider();
}

Runtime Trait

pub trait Trait {

  /// Working group worker identifer type
  type WorkerId: ...;

  /// Service Provider type.
  type ServiceProviderId

  /// Checks whether the given service provider id correspond to worker with given account as the role account
  fn is_service_provider_account(
        service_provider_worker_id: ServiceProviderWorkerId,
        account_id: AccountId,
    ) -> bool;

  /// Working group pallet integration.
  type WorkingGroup: common::working_group::WorkingGroupIntegration<Self>;

}

Protocol Operation

The protocol involves three different kinds of parties: gateway users, gateways and service providers. They interact through a mixture of off-chain and on-chain interactions, and the following sequence of scenarios attempts to capture how perviously sketched APIs and blockchain business logic is combined to make the system work.

1. Creating Gateway Users

A new prospective user and a given gateway must have some process by which they negotiate the access, and terms under which that access may take place, to the services of the network. This process will likely require some mechanism by which the gateway can gain sufficient assurance that the user can generate sufficient revenues to the gateway so as to cover the outlays the gateway is about to make on the users behalf. This entire process is outside this proposal, and it will invariably vary from gateway to gateway. When a new user is created, the gateway must inform all associated service providers about this new user. This is done through a dedicated message (ProviderSpecificUserProfileAdded) published on the message stream for service providers (serviceProviderEvents). The message payload is distinct for each provider, in particular must the authorization token (::profile::constraints::auth_token) be distinct across providers, otherwise it more easily facilitates abuse. If reused across profiles with same user, then users can free ride of each other. If reused across providers, then a single provider can resell the token to another gateway, allowing it to free ride off other gateways.

2. Gateway User Authentication

In order for a gateway user to access the service providers associated with the gateway, the user must learn about the set of service providers of various kinds that are associated with the gateway (UserLoggedInOnGatewayResult::user_specific_provider_profiles), and associated authorization tokens that will allow the user to get service from the providers. This information is initially learned by authentication with the gateway (login_user), which also has a message stream allowing each user to stay up to date with any changes in this information (gatewayUserEvents).

3. Service Channel Setup: Creation & Confirmation

Prelude

The gateway operator will, by some process, conclude that allowing users to access the services of a given provider under some terms is desirable. These terms are:

• What constitutes the unit of service, which is likely going to be subject to social convention depending on the type of service offered. So for example for bandwidth providers the unit could be 100 GB of data.
• The number of tokens going to the provider per unit of service. This value for all future channels for this provider exists in the chain state, and is updatable by the service provider.
• The number of token going to the platform per unit of service. This value for all future channels for this provider exists in the chain state, and is updatable by the gateway working group lead.
• The amount of capital that must be locked up up front.
• The time it takes to execute a unilateral refund by the gateway. This value for all future channels exists in the chain state, and is updatable by the gateway working group lead.

Despite some of these values existing in the chain already, its very likely that the gateway operator may want to communicate with the relevant parties in advance, possibly to change these values.

Channel Setup

The operator will in the end invoke an extrinsic (create_channel), committing to these values and the service provider. It is also possible to attach an SLA, which can also be augmented later by double opt-in.

This extrinsic will fail if there is already an existing channel with this provider, regardless of it's state, as there can only ever be one channel per combination of gateway and provider. The committed to values must also match the current on-chain values. If this is satisfied, then a new service channel (ServiceChannel) is created with the appropriate values, and in the ServiceChannelState.Pending state, and the capital is locked into an account that is not under the direct control of any actor.

The next step is for the service provider to confirm that they accept this service channel by simply invoking an extrinsic (confirm_channel) which transitions the channel to the ServiceChannelState.Operational state. Only at this point is it safe for the gateway to begin rendering service. Lastly, if the service provider does not confirm in a timely manner, the gateway can always instantly close the channel and recover the locked funds by a separate extrinsic (cancel_channel) so long as the channel is still in the ServiceChannelState.Pending state.

At this stage, the gateway and the provider are said to have an active channel, however, this does not imply that users of the gateway now can receive service from the provider, as this depends on whether the provider and gateway have agreed to actually allow users to use the service at the moment, a property called availability, reflected in the API.

The following figure depicts a successful interaction between the relevant parties.

Informing Gateway Users

Regardless of whether the channel is initially available or not, the gateway should inform all current active users by emitting a NewUserSpecificProviderProfile event over the stream gatewayUserEvents, and also list this profile in future calls to get_user_specific_provider_profiles.

4. User Status Update

A number of things may change in the system w.r.t. a given user that the user will want to learn about promptly, specifically

1. Availability of a provider changed: This would typically occur as a result of a mutual agreement between the gateway, provider and the working group lead.
2. Set of hosts associated with a provider changed: This can happen at any time and is publicly broadcasted on-chain.
3. The user may be individually blocked.
4. The authorisation token may be changed.
5. Resource utilisation limits may have changed (see parts of UserOnProviderAccessConstraints).

Whenever any of these are changed, the gateway should emit a UpdatedUserSpecificProviderProfile even over the stream gatewayUserEvents of the relevant user.

5. Provider-to-Gateway Authentication

The authentication follows the challenge response model, and goes as follows.

1. First the service provider makes an initial call (initiate_auth_challenge) providing service_provider_id to a gateway host.
2. As long as the gateway recognises the identifier as a valid provider, it should always yield a positive response, even if there is an on-going authentication process with the same provider. The response will include a unique nonce (AuthChallenge::nonce) that is never reused.
3. The service provider will then sign this nonce with their on-chain role key, and send this signature back in a second call (validate_auth_response).
4. The can validate the signature, in if valid should return a message SuccessfulAuthResponseValidation.

In principle there is no need for a service provider to authenticate with a gateway unless they have an active channel, but there is also no harm in allowing it, hence it is allowed.

6. Service Consumption

In this scenario there is no explicit API call for requesting and providing service (see request_service), as it would vary substantially with the service type in question, the general workflow is however the same regardless of type.

Gateway User Request

As the user is using the system, the user facing application will eventually detect that the user requires access to some resources that is provisioned by service providers. Some service provider type specific mechanism will be required to map this resource to the set of feasible service providers. From this it can be determined what subset of these also are servicing the gateway to which the user corresponds. For the purpose of subsequent steps, assume the gateway user has identifier denoted by gateway_user_id and a suitable service provider is has identifier denoted by service_provider_id. The user is able to obtain a set of suitable hosts corresponding to the service provider sing the information from the gateway (UserSpecificProviderProfile::service_provider::hosts), and a request for service is issued to the user API on this host, along with gateway_user_id, the authorisation token for this provider (UserSpecificProviderProfile::constraints::auth_token) and some service type specific resource identifier.

If the user is unable to successfully interact with the provider, then a report to that effect is issued to the gateway (report_service_provider_problem), otherwise the success is reported (report_service_consumed).

Service Provider Handling

The service provider must authenticate the user request and ensure that the user is not blocked and that the service with the gateway is the right availability. The service provisioning must be logged for future reporting purposes, and a payment request to the gateway may need to be triggered.

7. Service Channel: Cooperative Payment

When a service provider reaches a local tolerance limit for maximum outstanding service with a given gateway, it will request a payment signature (sign_payment) for a new level of total service. The gateway must then compare the request to the local estimate, and verify whether it is within agreeable error bounds, as well as apply any other suitable signing policy. The signature returned in the success case is to be used to settle the channel (settle_channel) with the given service level, as such it a signature with the key for the gateway fallback account over a hash digest of a message identifying each payment, namely concatenated serialization of field values in PaymentSignatureRequest and the channel id.

Implementation

Here we outline some notes that may be useful for planning the full implementation of this proposal.

Runtime

The runtime will require the following changes.

1. A new gateway working group.
2. Updating proposals + CLI to work with new gateway group.
3. The new specced gateway module.

Service Provider Node

Standalone vs Built-In

From an engineering point of view, a major question here is whether there should be a separate service provider node which handles the interaction with the gateway and the user, while being able to interact with different types of service nodes, as is implied will exist by the ServiceType type in the API. This has the benefit of allowing the core service node, such as Colossus for storage, to focus squarely on managing the service provisioning. Another alternative is to require each type of service node to wholesale replicate the service provider API described, and somehow reusing the basic gateway&user management logic and state in some other way.

Performance, Scalability and Fault Recovery

While it has not been clearly stated here, or anywhere else, performance and the ability to scale up with demand, is vital for service providers. The presentation of the storage node in this design has been from a logical point of view, the actual infrastructure architecture generating the described behaviour developed with a concern for these objectives. Likewise, both the API and this architecture must be informed by the goal to be able to recover from honest faults, something that must be expected under real world conditions.

Gateway Node: Charon

The long term trajectory to be expected is that operators will initially augment, and eventually roll their own gateway implementations entirely, in order to customise their own way of managing users and monetisation. However, in order to lower the barrier to entry during the early phase of the project, before operating Gateways has been demonstrated to be a viable enterprise, and also to offer a base reference implementation which others can build upon, we must build a sort of initial least common denominator gateway, herein called Charon.

Starting with the most basic parts required for the gateway protocol, and then also adding the user management and eventually monetisation through powering Atlas ad-placements could be in scope.

Performance, Scalability and Fault Recovery

See same section above for the service provider node.

Atlas

The situation here is likely going to be similar to the gateway node in terms of its purpose and likely future after mainnet. It should be built out in concert with the changes on the gateway side.

Report Format

As indicated by the API, and also by the reporting to the council by service providers in the schema above, there needs to be a standardised format to report claims about what resources users of a gateway have consumed from a given service provider over some period of time. The purpose of such reports is in general to support ex-post forensics of fraud, abuse and also parceling out reward payments to creators based on traffic generated.

Orion Reporting & Integration

Currently our approach to managing viewership and follower statics and metrics, through the Orion node, has had no clear path for working in a production environment. The introduction of gateways may offer a more clear path by resting on the natural screening function that gateways engage in. Specifically, gateways can act as a proxy for the actions and events triggered by their users, and relay them to the Orion node. This at least provides a singular unified data source for this information across all gateways, and also discourages abusive users, but of course still has other reliability and trust issues. These may however be sufficiently benign given the low direct value of abusing this information.

Attacks

This proposal has many attacks, mostly just stemming from collusion between two parties at the expense of third, but none of them are non-starters.

┆Issue is synchronized with this Asana task by Unito