Upgrade to @polkadot/api ^1.25.1 and required changes

This is a summary-issue that will later be split into smaller issues w.r.t. to specific projects like @joystream/types, CLI and Pioneer.

Sources

• https://github.com/polkadot-js/api/blob/master/CHANGELOG.md
• https://github.com/polkadot-js/api/blob/master/UPGRADING.md

Expected changes

Substantial changes:

• All types' constructors inside @joystream/types now need to take Registry as first argument, ie.:

export class BlockAndTime extends Struct {
    constructor (value?: BlockAndTimeType) {
        /* ... */
    }

Needs to be changed to:

export class BlockAndTime extends Struct {
    constructor (registry: Registry, value?: BlockAndTimeType) {
        /* ... */
    }

Perhaps that's a good opportunity to normalize all struct to use JoyStruct, all enums to use JoyEnum etc., so we can more easily implement functionality that we need for all structs/enums (ie. helpful getters, better type-safety) and be better prepared for future changes that affect those (although more research is required in order to determine how this will work w.r.t. https://polkadot.js.org/api/examples/promise/90_typegen/ and whether those are mutually exclusive or not)

• Related to the above change: We can no longer use constructs like new BlockAndTime(), but should use api.createType() instead. This will probably require quite a lot of changes in the codebase.
• Breaking change (TypeScript only) The *.d.ts files now contain TypeScript 3.8 features (specifically #private), which is not usable in TS versions < 3.8 - we already ran into some issues when trying to upgrade to TS > 3.8 in the past so I think this may require some more substantial changes/fixes in our TypeScript projects.
• There are a few changes in api.derive.staking that may affect packages like app-staking and components like react-components/src/AddressInfo.tsx in Pioneer
• We should now use use .entries(), .keys() etc. to fetch map/double-map entires (linked maps will not longer be available)

Smaller changes:

• U8a and Data types from polkadot/types/codec have been renamed to Raw

T.B.D.

• Important Chains without balances.account or system.account should supply "BalanceLock": "BalanceLockTo212" to their types to use the previous-generation BalanceLock (when querying locks)
• Important Update RPC status codes (latest Substrate 2), with the isInBlock (isFinalized now indicates finality)
• Breaking change Session Keys defaults to 4 keys now (Substrate master), depending on your node config, you would want to override with e.g. Keys: 'SessionKeys3' (1, 2, 3, 4, 5 & 6)
• Breaking change ValidatorPrefs now default to the new percentage commission model, to use the previous validatorPayment, you can override with ValidatorPrefs: 'ValidatorPrefsTo196'
• Important Substrate master has updated staking, on older chains supply StakingLedger: 'StakingLedgerTo223'
• Important Substrate master has updated collective, on older chains supply Votes: 'VotesTo230'
• Breaking change api.rpc.state.queryStorage(...) now fully decodes the Vec<StorageChangeSet> and returns a decoded [Hash, Codec[]][] when using this RPC.
• Breaking change The format for any custom RPCs have been changed (alongside API-internal changes) to allow for better RPC management. If you are currently using custom RPCs (or planning to do so), look at the updated documentation
• Breaking change Alongside API RPC changes, the @polkadot/jsonrpc package has been removed. Since it was never documented and only used internally, this should not have adverse impacts. All RPC definitions itself has now been moved to the relevant modules inside @polkadot/types/interfaces
• Important Substrate has an updated democracy module. If using an older chain add the ReferendumInfo: 'ReferendumInfoTo239' type when using referendums
• Breaking change (TypeScript with derives only) The return type names for api.derive.* have been adjusted for consistency, all starting with Derive* (previously a mixture of Derived* and Derive*)
• Breaking change (derives only) Additional cleanups of democracy proposal & referenda derives, including support for new Polkadot/Substrate vote retrievals
• Important Substrate master has updated staking, on older chains apply StakingLedger: 'StakingLedgerTo240'
• Important Substrate master has an updated Weight (to u64), on older chains apply Weight: 'u32' (This does affect event decoding if wrongly set)
• Important DispatchInfo changed paysFee to an enum, if using an older version use, DispatchInfo: 'DispatchInfoTo244'
• Important RPC for account_nextIndex has been moved to system_accountNextIndex (Aligning with Substrate as a primary alias)
• Important Substrate changed the treasury tip structure, on older chains supply OpenTip: 'OpenTipTo225'
• Important Contract RPC result updated on Substrate, on older chains supply ContractExecResult: 'ContractExecResultTo255'

Possible improvements:

• We can now use https://polkadot.js.org/api/examples/promise/90_typegen/ for api TypeScript decoration based on the runtime we connect to (that means ie. auto-suggesting modules, methods, arguments, "return" values etc. for api.tx, api.query and api.consts (but this can only be truly tested after we have a working runtime)
• We can use BTreeSet implementation from @polkadot/types (instead of our own)
• Use paymentInfo in CLI's fee calculation (minor)

┆Issue is synchronized with this Asana task by Unito

Start by reading the linked polkadot-js API CHANGELOG and UPGRADING.md, then split the work across @joystream/types, CLI, Pioneer, and the affected TypeScript projects. Inspect react-components/src/AddressInfo.tsx and the staking, map-query, and type-constructor usages mentioned in the issue; done requires separate scoped changes with compatibility verified across those projects.