Metaprotocol DevUX

Background

We are exploring using metaprotocols for some, possibly many, parts of the Joystream system, see idea here

https://github.com/Joystream/joystream/issues/1990

and an example application to the content directory here

https://github.com/Joystream/joystream/issues/1991

DevUX goal

mnaamani already pointed out that under this scheme, when submitting a metatransaction, the normal flow where the full node you submitted the transaction to telling you whether it worked out or not does not work, because you still need to know whether the metatransaction will be accepted by the metaprotocol nodes (query node in this case). So how should this be faciliated in a way which gives a convenient devux. Moreover ,there is also the question of how you allow apps to hook into metaprotocol events, even when they did not make the transaction. This is likely very important for lots of use cases.

So you want something like

let joystream_lib_instance = new JoystreamLib(full_node:..., query_node: ...);

joystream_lib_instance.on(`VideoPublished`, (channel) => {
  // yay lets do something fun here
});

/* cancel using handle if desired ? */
let handle = joystream_lib_instance.createChannel(channelCreationParmeters, (result, err) => {

 if(err) {
   /// nooooooo we failed.
 }

})

where the assumption is that channel creation and publishing videos under channels are metaprotocol transactions.

Proposal

Here is one way to try to achieve this.

Assumptions

Let us assume

1. Each extrinsic only has a single metatransaction in it. If one wanted more flexibility one could simply make metransactions that have more complex payloads
2. There is a dedicated extrinsic on-chain where members can submit metaprotocol messages, so it is something like
   Metaprotocol::sendMessage(origin, member_id: MemberId, serialized_message: vec<u8>)
3. There is only a single metaprotocol transaction CreateChannel available, perhaps described by

message CreateChannel {
  required string title;
  required int32 cover_photo_data_object_id;
}

1. Hydra supports entity subscriptions https://github.com/Joystream/hydra/issues/159.
2. The query node has something like

type CreateChannel @variant {

	member_id: BigInt!

	title: String!

	cover_photo_data_object_id: Int!
}

type Invalid @variant {
	/* nothing */
}

union DeserializedMetaTransaction = Invalid | CreateChannel

type Metatransaction @entity @with_subscriptions {

	# Id of base extrinsic carrying payload
	extrinsic_id: Bytes! @unique

	# Block
	block_height: BigInt!

	# metatransaction type
	deserialized_transaction: DeserializedMetaTransaction!
}

Caller feedback

Either early in the lifecycle of createChannel, or potentially when the JoystreamLib instance is created, a subscription to the resulting Metatransaction message stream in the query node API must be established. Regardless of which of the two, it just needs to be established fully prior to any extrinsic being issued, otherwise one can end up in a race condition. Then JoystreamLib::createChannel has to build this message, serialize, compose the Metaprotocol::sendMessage extrinsic, get it signed, and send it to the full node. If the extrinsic itself fails, then the javascript function must immediately inform the developer with err result about this. If it does not fail, then createChannel will just has to wait for the appropriate message to arrive on the subscription channel, by just filtering on the relevant extrinsic_id for example. When the conclusion arrives, this result can be relayed back to the caller in whatever form is suitable in the result.

Events

The JoystreamLib isntance just uses an event emitter to relay some specific event when it arrives from the same subscription endpoint.

Notes

• A big side-benefit of this proposal is that we get an actual block explorer like API for the metaprotocol itself through the Metatransaction entity, as it will be very impractical to try to look directly at the
• We are here sidestepping the question of how to gracefully integrate the client side signing interaction, as it is somewhat unrelated to the core question of metaprotocol issue.

Questions

• Too complex?
• We need new subscription feature in Hydra, which depends on Warthog changes and Hydra changes, and the former appears unclear when will be ready and merged.
• Is there a better temporary solution?
• Does it need to be this convenient? other options?

┆Issue is synchronized with this Asana task by Unito