Feature: Procedures

Introduction

A "procedure" (named vaguely to invoke the idea of RPC) is a command that can be submitted to ShareDB, where the server (rather than the client) will produce and submit the ops. The client may optimistically generate and apply its own ops locally, but these should be discarded in favour of the server's implementation when it responds. An example procedure call might look something like:

doc.submitProcedure('command', payload, (error) => {});

Motivation

The primary benefit of this feature is to move op generation server-side. This would be useful where:

• The client is not "trusted" to create some ops correctly (anything from adding a timestamp to an op to creating the correct structure for a particular schema)
• Making changes to how ops are produced without updating clients
• The client does not have enough information to construct an op (eg it would have to make a server call anyway to get some information needed for the op)
• Generating the op is expensive, and might be faster if executed remotely
• Ops are highly coupled to the current doc.data, and should be radically different when reconnecting after a long time offline (eg "find and replace" in a text document, whose content has changed)
• We want to guarantee certain properties by centralising op generation (eg generating unique ID fields)

Implementation

The implementation of procedures is both simple and complex at the same time.

If we don't want the client to generate optimistic ops, then the implementation is relatively trivial: we just need to add a way of getting a command and payload to the server, where the server generates its own "client" and submits ops upon receipt of the command. Our own client will eventually receive the submitted ops and apply them.

Applying an optimistic local procedure is far trickier.

Why bother with optimistic procedures?

Firstly, submitOp already optimistically applies ops, so we'd want to stay as consistent with that as possible. We (presumably) apply local ops optimistically, because:

• We can (it's one of the benefits of OT)
• It gives a much faster response and smoother experience

Note that when consumers implement procedures, providing an optimistic op for local consumption will be optional anyway (eg in case the client absolutely needs information from the server before updating).

What's so difficult?

The key conceptual difference between submitOp and submitProcedure is this: we assume that the local op applied submitOp is canonical, whereas we assume the local op applied by submitProcedure is **not ** canonical (otherwise why bother with this at all?).

Rollback

We naturally already deal with non-canonical ops being submitted by submitOp: this takes us down the ERR_OP_SUBMIT_REJECTED pathway, which attempts to roll the document back to a useable state:

• First by attempting to invert our inflight op
• And then by performing a "hard" rollback — resetting the Doc state (including any pending ops we had)

This is okay as a recovery strategy for a failed submitOp. However, we'd have to assume that submitProcedure would always take us down this path as its mainline pathway, which it's not so appropriate for, because:

• We shouldn't assume that types are invertible, since invert is an optional method
• A "hard" rollback is obviously out of the question for a mainline use case, because we'd regularly bin pending ops

We could potentially adapt the rollback method:

• Clone doc.data before optimistic apply (although this could be expensive)
• Cache incoming ops
• When we receive acknowledgement of the procedure (and the associated canonical op), we reset data to our cloned version and replay all cached ops.

However, even doing this takes us to a slightly weird place with events.

Events

One of the other weird things about an optimistic procedure apply is that — when we do it — we emit an op event that we assume is not canonical. Anyone listening for the op events and attempting to replay these events on a mirrored object (say), should always end up in a state consistent with Doc.data

Again, in the rollback case this is handled by:

• trying to invert, and then emitting the inverted op, which keeps our listener consistent with data
• falling back to a hard rollback, where we emit a load event upon ingesting the snapshot

This leads to some questions:

• Do we expect clients to handle the load event everywhere they handle op? Often you can get away without handling load, because you just wait for the subscribe callback, where your data has been initialised
• Is load considered to be a "happy" path?

Possible approaches

"Less hard" rollback

As outlined above, we could tweak the "hard" rollback behaviour to:

• Clone doc.data before optimistic apply (although this could be expensive)
• Cache incoming ops
• When we receive acknowledgement of the procedure (and the associated canonical op), we reset data to our cloned version and replay all cached ops.

Advantages:

• Leverages a lot of existing code
• Works with types that aren't invertible
• Invertible types can still use invert to simplify things
• Improves hard rollback for submitOp as a byproduct

Disadvantages:

• Assumes all clients handle the load event
• Expensive clone operation, which is also inconsistent with the general approach taken in ShareDB, which usually attempts to avoid cloning
• Feels a bit "brute force"-ish for non-invertible types

Batch op events until procedure acknowledgement

We could potentially wait to emit the procedure op event until the canonical op is returned from the server. We would still need to combine this with the "less hard rollback" approach.

Advantages:

• No need to handle load event (even though we'd still re-ingest a snapshot internally)

Disadvantages:

• Inconsistent with submitOp behaviour (which is synchronous)
• There's a window where a mirrored object watching for op events would drift from doc.data, because we've optimistically applied in doc.data without emitting an op event

Delegate optimistic recovery to the procedure

One potential approach is to completely move the responsibility of returning us to a canonical state from ShareDB into the procedure. For example, a procedure may be given the optimistic version the client applied, and it's the procedure's duty to either:

• Invert the optimistic op; or
• Calculate a diff between its own op and the optimistic op

Advantages:

• Relatively easy to implement in ShareDB
• Don't have to worry about cloning snapshots, or inverting ops

Disadvantages:

• Much harder to develop procedures
• Generally not a nice consumer API

Start with the existing submitOp and ERR_OP_SUBMIT_REJECTED rollback paths, then trace how op and load events keep mirrored objects consistent. The issue names no files or tests; progress requires choosing and documenting the procedure API, server-side op generation, optimistic reconciliation, and event semantics, followed by tests for the selected behavior.