hyperledger / hyperledger/fabric-x-common
[blk storage] Adopt the Orderer optimizations for the block storage
- Dominant language
- Go
- Stars
- 21
- Forks
- 15
- Avg merge
- 2d 19h
- Merged PRs (30d)
- 20
Description
The Fabric-X Orderer carries its own copy of this package at
[`common/ledger/blkstorage`](https://github.com/hyperledger/fabric-x-orderer/tree/main/common/ledger/blkstorage).
It is a sibling fork, not a descendant — both trees come from Hyperledger Fabric's `blkstorage`, and
each has diverged since. `cache.go`, `block_stream.go`, `config.go`, `metrics.go` and
`protobuf_util.go` are still byte-identical; the rest differs in both directions.
The divergence that matters here has one cause. All three orderer ledgers — assembler, batcher, and
consenter — configure their store with `AttrsToIndex: []IndexableAttr{IndexableAttrBlockNum}` and
nothing else, so no transaction ID index exists anywhere in the orderer. The same is true of a
Fabric-X Committer sidecar with the transaction ID index disabled
(hyperledger/fabric-x-committer#784). Both therefore pay for per-transaction index information that
nothing ever reads. The orderer's fork stopped paying; this repo's copy still does.
This umbrella tracks porting those wins back. Three, in descending order of measured impact:
1. **#165** — do not build transaction index information no index will read, on the **write** path
(`serializeBlock`). Addressed by #166, which is a superset of the orderer's version: it derives the
decision from the index configuration rather than a caller-passed bool, also skips the offsets and
not just the transaction IDs, and pre-sizes the serialization buffer. Nothing left to port.
2. **#182** — the read path — `extractData` computes a transaction ID per envelope on every block read and
`deserializeBlock` discards all of it. Not touched by #166. This is the same unmarshal as #165, on
the other side of the store, and unlike #165 it is wasted unconditionally rather than only for
stores without a transaction ID index.
3. **#183** — header-only reads — `retrieveBlockHeaderByNumber` and `retrieveFirstBlockNumFromFile` parse
every envelope and all metadata to read a three-field header.
Not in scope, and not to be adopted: the orderer's fork dropped snapshot bootstrap and export,
`reset` and `rollback`, and left `TODO`s where the snapshot front-boundary guards used to sit,
marking them for ledger pruning. This repo is also ahead of the orderer on `AddBlockNoSync`/`Flush`
with a sync before file rollover (#69), the `context`-aware `blocksItr` (#88), the `ErrNotFound`
sentinel (#90), and `cockroachdb/errors`. Each item below is therefore additive to this tree — there
is no orderer behaviour to take wholesale.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.