hyperledger / hyperledger/fabric-x-common

[blk storage] Adopt the Orderer optimizations for the block storage

Open
#167 0 comments 0 reactions 1 assignee Claimed by @liran-funaro View on GitHub
enhancement performance
Dominant language
Go
Stars
21
Forks
15
Avg merge
2d 19h
Merged PRs (30d)
20

Description

The Fabric-X Orderer carries its own copy of this package at
[`common/ledger/blkstorage`](https://github.com/hyperledger/fabric-x-orderer/tree/main/common/ledger/blkstorage).
It is a sibling fork, not a descendant — both trees come from Hyperledger Fabric's `blkstorage`, and
each has diverged since. `cache.go`, `block_stream.go`, `config.go`, `metrics.go` and
`protobuf_util.go` are still byte-identical; the rest differs in both directions.

The divergence that matters here has one cause. All three orderer ledgers — assembler, batcher, and
consenter — configure their store with `AttrsToIndex: []IndexableAttr{IndexableAttrBlockNum}` and
nothing else, so no transaction ID index exists anywhere in the orderer. The same is true of a
Fabric-X Committer sidecar with the transaction ID index disabled
(hyperledger/fabric-x-committer#784). Both therefore pay for per-transaction index information that
nothing ever reads. The orderer's fork stopped paying; this repo's copy still does.

This umbrella tracks porting those wins back. Three, in descending order of measured impact:

1. **#165** — do not build transaction index information no index will read, on the **write** path
(`serializeBlock`). Addressed by #166, which is a superset of the orderer's version: it derives the
decision from the index configuration rather than a caller-passed bool, also skips the offsets and
not just the transaction IDs, and pre-sizes the serialization buffer. Nothing left to port.
2. **#182** — the read path — `extractData` computes a transaction ID per envelope on every block read and
`deserializeBlock` discards all of it. Not touched by #166. This is the same unmarshal as #165, on
the other side of the store, and unlike #165 it is wasted unconditionally rather than only for
stores without a transaction ID index.
3. **#183** — header-only reads — `retrieveBlockHeaderByNumber` and `retrieveFirstBlockNumFromFile` parse
every envelope and all metadata to read a three-field header.

Not in scope, and not to be adopted: the orderer's fork dropped snapshot bootstrap and export,
`reset` and `rollback`, and left `TODO`s where the snapshot front-boundary guards used to sit,
marking them for ledger pruning. This repo is also ahead of the orderer on `AddBlockNoSync`/`Flush`
with a sync before file rollover (#69), the `context`-aware `blocksItr` (#88), the `ErrNotFound`
sentinel (#90), and `cockroachdb/errors`. Each item below is therefore additive to this tree — there
is no orderer behaviour to take wholesale.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.