New BSIP: Make Update Operation Fields Optional

BSIP: TBD
Title: Make Update Operation Fields Optional
Authors: Nathan Hourt
Status: Draft
Type: Protocol
Created: April 6, 2019
Discussion: https://github.com/bitshares/bsips/issues/159

# Abstract
The BitShares protocol specifies a number of operations to update objects within the database. Many of these operations contain several fields, all of which must be specified, even if only one or a few of them are being updated. This requires the operation to redundantly specify old values of fields for which no change is desired, which is wasteful of valuable blockchain storage space. Moreover, this strategy makes stateless analysis of such updates, i.e. to discern which fields are being modified, impossible.

This BSIP proposes to replace object update operations which contain many non-optional fields with new versions which specify all updated fields optionally. This will reduce the storage size of affected operations and make stateless analysis of their effects possible. Following a grace period, the old operations may be deprecated to require usage of the newer, more efficient versions.

# Motivation
The size of the BitShares blockchain history is growing rapidly. Moreover, the rate of usage is increasing, causing the growth to accelerate. While the long term solution is the development of improved storage and handling of historical data, in the meantime it behooves stakeholders to eliminate frivolous waste of storage space. Several of the database update operations currently require the superfluous specification of all fields rather than only the particular fields the user wishes to modify. This results in storage space being consumed by redundant information which provides no benefit whatsoever to the stakeholders or software.

Another compelling motivation to adopt this change is that it supports other desirable use cases. BSIP 40 is a proposal to add a new authority type to the blockchain, the "Custom Active Authority," which grants limited privileges to sub-authorities rather than requiring full active authority for all operations. One important use case for this feature is to allow the creation of a limited authority capable of updating, for example, an account's memo key, but not its votes. With the current design of `account_update_operation`, however, an update to the memo key must also specify the votes, and it cannot be determined whether the new votes are identical to the old ones without accessing chain state.

As currently specified, BSIP 40 cannot provide for this use case, and modifying the proposal to support analysis of the operations with access to chain state increases its implementation complexity dramatically and may have detrimental impacts on future blockchain scalability improvements. The addition of new update operations that specify field updates optionally is a comparatively small modification to the blockchain protocol, and it enables BSIP 40 to support this critical use case with no alterations to its current specification.

# Rationale
BitShares operations are designed according to some loosely defined principles, one of which is that they should facilitate stateless analysis of their effects. For instance, operations which modify an account balance are expected to explicitly state the amount and asset type being moved, even when these can be inferred from chain state, so that programs can track the balance of accounts by looking at operation history alone, without requiring access to chain state. This principle can also be extended to update operations to require that such operations facilitate stateless analysis of their changes -- in particular, the ability to determine which specific fields are being changed.

An update to a field can be expressed in two ways: a delta of change, or a replacement value. Thus the question is raised, which is more important to stateless analysis -- the delta of change, or the new value? This proposal suggests to use the new value for simple field updates, but adjustment of asset balances must always be specified as a delta, as these adjustments represent movement of asset rather than arbitrary specification of a value. Updates to sets will support both a delta (items to add/remove) and a new value (complete replacement set), with the requirement that only one of these options be specified in a given operation.

The complete set of update operations presently defined by the protocol is as follows:
- `account_update_operation`
- `asset_update_operation`
- `asset_update_bitasset_operation`
- `asset_update_feed_producers_operation`
- `asset_update_issuer_operation`
- `call_order_update_operation`
- `committee_member_update_operation`
- `committee_member_update_global_parameters_operation`
- `proposal_update_operation`
- `withdraw_permission_update_operation`
- `witness_update_operation`
- `htlc_extend_operation`

Several of these operations are already complicit with the above principle. Only the following require alteration or replacement to comply:
- `account_update_operation`
- `asset_update_operation`
- `asset_update_bitasset_operation`
- `committee_member_update_global_parameters_operation`
- `withdraw_permission_update_operation`

Note: `call_order_update_operation` is an edge case, as it requires specification of both `delta_collateral` and `delta_debt`; however, these are deltas and therefore support stateless analysis, and the size advantage gained by specifying only one of these values versus both of them is not deemed sufficient to justify replacing and deprecating the operation.

# Specification
Two hardfork dates will be specified for the implementation of this BSIP: one is the activation date for the new operations/extensions, and the other is the cutoff date for the old operations/fields.

To reduce boilerplate around set updates, which may be expressed either as a delta or a complete replacement set, the following template will be defined:

```
template
struct set_updater {
// A delta is a map of Elements to a bool where true means add the element, and false means remove it
using delta = flat_map;
using new_set = flat_set;
optional> update;
};
```

A discussion of the proposed modifications are described for each affected operation below.

### Account update
Of the operations requiring alteration, `account_update_operation` stands out since all of its update fields are already optional; however, one field in the operation, `new_options`, is an object with several non-optional sub-fields.

Since all fields on the operation itself are already optional, the existing operation does not need to be deprecated; instead, new operation extensions can be defined which update each sub-field in `new_options` individually, and `new_options` can subsequently be required to be a null optional after the cutoff date. Extensions are always optional, thus satisfying the optional updates design principle.

The `account_update_operation::new_options` field is an optional of type `account_options`. The fields of `account_options` are:

```
struct account_options {
public_key_type memo_key;
account_id_type voting_account;
uint16_t num_witness;
uint16_t num_committee;
flat_set votes;
};
```

The following extensions will be added to `account_update_operation` to support updating these fields without using `new_options`:

```
optional new_memo_key;
optional new_voting_account;
optional new_num_witness;
optional new_num_committee;
set_updater votes_update;
```

The evaluator will be updated to verify that, if `num_witness` or `num_committee` or the votes are updated, then the desired number of witness/committee seats does not exceed the number of votes.

### Asset Updates
Of the operations requiring alteration, the `asset_update_operation` also stands out, as it only has two update fields, one of which, `new_issuer`, is already optional, but is also required to be null as it has already been replaced with the dedicated `asset_update_issuer_operation`. The other field is a struct called `asset_options`. The fields in this struct are listed below:

```
struct asset_options {
share_type max_supply;
uint16_t market_fee_percent;
share_type max_market_fee;
uint16_t issuer_permissions;
uint16_t flags;
price core_exchange_rate;
flat_set whitelist_authorities;
flat_set blacklist_authorities;
flat_set whitelist_markets;
flat_set blacklist_markets;
string description;

};

{
uint16_t reward_percent;
flat_set whitelist_market_fee_sharing;
};
```

A new `asset_options_update_operation`, which does not contain the deprecated `new_issuer` field, will be defined with the fields specified below. After the cutoff date, the `asset_update_operation` will be deprecated and disabled from use.

```
struct asset_options_update_operation {
/* asset to update, fee, payer... */

optional new_max_supply;
optional new_market_fee_percent;
optional new_max_market_fee;
optional new_issuer_permissions;
optional new_flags;
optional new_core_exchange_rate;
set_updater whitelist_authorities_update;
set_updater blacklist_authorities_update;
set_updater whitelist_markets_update;
set_updater blacklist_markets_update;
optional new_description;
optional new_reward_percent;
set_updater whitelist_market_fee_sharing_update;
};
```

A second asset updating operation, `asset_update_bitasset_operation`, must be modified as well. This operation bundles all of its fields for updating into a single struct field of type `bitasset_options`:

```
struct bitasset_options {
uint32_t feed_lifetime_sec;
uint8_t minimum_feeds;
uint32_t force_settlement_delay_sec;
uint16_t force_settlement_offset_percent;
uint16_t maximum_force_settlement_volume;
asset_id_type short_backing_asset;
};
```

A new operation, `asset_update_bitasset_options_operation`, will be defined with the fields listed below. After the cutoff date, `asset_update_bitasset_operation` will be deprecated and disabled from use.

```
struct asset_update_bitasset_options_operation {
/* asset to update, fee, payer... */

optional new_feed_lifetime_sec;
optional new_minimum_feeds;
optional new_force_settlement_delay_sec;
optional new_force_settlement_offset_percent;
optional new_maximum_force_settlement_volume;
optional new_short_backing_asset;
};
```

### Global Parameters Update
The `committee_member_update_global_parameters_operation` contains all of its updates in the `chain_parameters` struct:

```
struct chain_parameters {
std::shared_ptr current_fees;
uint8_t block_interval;
uint32_t maintenance_interval;
uint8_t maintenance_skip_slots;
uint32_t committee_proposal_review_period;
uint32_t maximum_transaction_size;
uint32_t maximum_block_size;
uint32_t maximum_time_until_expiration;
uint32_t maximum_proposal_lifetime;
uint8_t maximum_asset_whitelist_authorities;
uint8_t maximum_asset_feed_publishers;
uint16_t maximum_witness_count;
uint16_t maximum_committee_count;
uint16_t maximum_authority_membership;
uint16_t reserve_percent_of_fee;
uint16_t network_percent_of_fee;
uint16_t lifetime_referrer_percent_of_fee;
uint32_t cashback_vesting_period_seconds;
share_type cashback_vesting_threshold;
bool count_non_member_votes;
bool allow_non_member_whitelists;
share_type witness_pay_per_block;
uint32_t witness_pay_vesting_seconds;
share_type worker_budget_per_day;
uint16_t max_predicate_opcode;
share_type fee_liquidation_threshold;
uint16_t accounts_per_fee_scale;
uint8_t account_fee_scale_bitshifts;
uint8_t max_authority_depth;

uint32_t max_timeout_secs;
uint32_t max_preimage_size;
};
```

A new operation, `global_parameters_update_operation` will be defined with the fields listed below. After the cutoff date, `committee_member_update_global_parameters_operation` will be deprecated and disabled from use.

```
struct global_parameters_update_operation {
asset fee;

flat_set fee_updates;
optional new_block_interval;
optional new_maintenance_interval;
optional new_maintenance_skip_slots;
optional new_committee_proposal_review_period;
optional new_maximum_transaction_size;
optional new_maximum_block_size;
optional new_maximum_time_until_expiration;
optional new_maximum_proposal_lifetime;
optional new_maximum_asset_whitelist_authorities;
optional new_maximum_asset_feed_publishers;
optional new_maximum_witness_count;
optional new_maximum_committee_count;
optional new_maximum_authority_membership;
optional new_reserve_percent_of_fee;
optional new_network_percent_of_fee;
optional new_lifetime_referrer_percent_of_fee;
optional new_cashback_vesting_period_seconds;
optional new_cashback_vesting_threshold;
optional new_count_non_member_votes;
optional new_allow_non_member_whitelists;
optional new_witness_pay_per_block;
optional new_witness_pay_vesting_seconds;
optional new_worker_budget_per_day;
optional new_max_predicate_opcode;
optional new_fee_liquidation_threshold;
optional new_accounts_per_fee_scale;
optional new_account_fee_scale_bitshifts;
optional new_max_authority_depth;
optional new_htlc_max_timeout_secs;
optional new_htlc_max_preimage_size;
};
```

### Withdraw Permission Update
The fields of `withdraw_permission_update_operation` are listed below.

```
struct withdraw_permission_update_operation {
asset fee;
account_id_type withdraw_from_account;
account_id_type authorized_account;
withdraw_permission_id_type permission_to_update;

asset withdrawal_limit;
uint32_t withdrawal_period_sec;
time_point_sec period_start_time;
uint32_t periods_until_expiration;
};
```

A new operation, `withdraw_permission_update_options_operation`, will be defined with the fields listed below. After the cutoff date, `withdraw_permission_update_operation` will be deprecated and disabled from use.

```
struct withdraw_permission_update_options_operation {
/* fee, payer, permission specifier... */

optional new_withdrawal_limit;
optional new_withdrawal_period_sec;
optional new_period_start_time;
optional new_periods_until_expiration;
};
```

# Discussion and Summary for Shareholders
The main cost of accepting this proposal is the addition of four new operations to the protocol, and the deprecation of four old ones, as well as the addition of several extensions to one operation and the deprecation of one of its fields. In return, however, the new operation versions will be much smaller, which is advantageous in the face of rapidly accelerating blockchain history growth. It should be noted, however, that these operations are not major contributors to the chain growth rate, and thus this improvement will be minor and will not substantially decrease the rate of growth, nor the acceleration of growth.

The main advantage gained by accepting this proposal is that the new operation versions are more self-describing. It will be possible to determine exactly which fields are updated merely by examining the operations themselves, rather than needing to simultaneously examine the blockchain database. This paves the road for critical use cases within the already-accepted BSIP 40 proposal, which are otherwise unsupportable.

# Copyright
Intellectual Property is BS.
Those who wish to pretend otherwise are hereby directed to treat this document as public domain.

# See Also
[BSIP 40](https://github.com/bitshares/bsips/blob/master/bsip-0040.md)

Start with the Specification section and review the listed affected operations, especially account_update_operation and the proposed asset update operations. Compare the proposal with the current BitShares protocol definitions; done means the draft resolves the optional fields, replacement operations, and activation and cutoff behavior it describes.

Written by the indexing model from the issue text.