hyperledger / hyperledger/fabric-x-common

[blkstorage] Do not compute transaction IDs when deserializing a block

Open
#182 0 comments 0 reactions 0 assignees View on GitHub
enhancement performance
Dominant language
Go
Stars
21
Forks
15
Avg merge
2d 19h
Merged PRs (30d)
20

Description

Part of the umbrella #167. This is the read-path half of #165: that issue removed transaction ID
computation from `serializeBlock`; the same unmarshal is still paid on every block read.

`extractData` (`common/ledger/blkstorage/block_serialization.go:134`) calls
`protoutil.GetOrComputeTxIDFromEnvelope` for every envelope and builds a `txindexInfo` and a
`locPointer` for each, unconditionally. `GetOrComputeTxIDFromEnvelope` unmarshals the whole envelope
and its payload header, so the cost is the same one #165 measured at 7.4% of the sidecar process on
10,000-transaction blocks — only here it is on the read side.

`deserializeBlock` (`block_serialization.go:45`) then discards all of it:

```go
if block.Data, _, err = extractData(b); err != nil {
```

Every caller of `deserializeBlock` throws the offsets away, so this is not an
"only some stores need it" question the way #165 was — the work is unconditionally wasted:

- `blocksItr.Next` (`blocks_itr.go:117`) — block delivery and streaming, per block served
- `blockfileMgr.fetchBlock` (`blockfile_mgr.go:659`) — behind `retrieveBlockByNumber`,
`retrieveBlockByHash`, `retrieveBlockByTxID`
- `constructBlockfilesInfo` (`blockfile_helper.go:70`) — the startup scan of the last block file
- `reset.go:113` and the rollback scans

The remaining caller of `extractData` that does read the offsets is
`extractSerializedBlockInfo` (`block_serialization.go:56`), used by `syncIndex`
(`blockfile_mgr.go:476`), `retrieveBlockHeaderByNumber` (`:604`), `rollback.go:137` and `:222`, and
`retrieveFirstBlockNumFromFile` (`blockfile_helper.go:127`). Of those, only `syncIndex` needs the
transaction IDs, and only when the store indexes by transaction ID — the same
`blockIndex.serializationNeeds` question #165 introduced for the write side applies here.

Proposal: give `extractData` (and `extractSerializedBlockInfo`) an `indexNeeds`, mirroring
`serializeBlock`. `deserializeBlock` asks for nothing, `syncIndex` asks the index, and the rollback
and file-scan paths ask for the offsets alone. The Fabric-X Orderer's fork of this package already
does the coarse version of this — a bool threaded through `extractData` — so the shape is known to work.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with common/ledger/blkstorage/block_serialization.go, especially deserializeBlock, extractData, and extractSerializedBlockInfo, then trace the callers listed in blocks_itr.go, blockfile_mgr.go, blockfile_helper.go, and rollback.go. Compare the existing serializationNeeds handling and the Fabric-X Orderer fork. Done means read paths avoid transaction-ID computation unless the caller's index requirements need it, while offset-dependent paths retain their behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
databases
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.