ipfs / ipfs/boxo

[ipfs/go-bitswap] Proposal: Streaming GetBlocks

Open
#121 1 comment 0 reactions 1 assignee Claimed by @lidel View on GitHub
Dominant language
Go
Stars
316
Forks
163
Avg merge
4d 12h
Merged PRs (30d)
8

Description

At the moment, it's a bit tricky to continuously prefetch blocks as one needs to launch a goroutine per batch.

Proposal: Change the signature of `GetBlocks` to take an input channel and return an output channel:

```go
// Fetcher is an object that can be used to retrieve blocks (defined in go-ipfs-exchange-interface)
type Fetcher interface {
// GetBlock returns the block associated with a given key.
GetBlock(context.Context, cid.Cid) (blocks.Block, error)

// GetBlocks returns a stream of blocks, given a stream of CIDs. It will
// return blocks in any order.
//
// To wait for all remaining blocks, close the CID channel and wait for
// the blocks channel to be closed. A closed channel does not mean that
// _all_ blocks were retrieved, it just means that the fetcher is done
// retrieving blocks.
GetBlocks(context.Context, <-chan cid.Cid) (<-chan blocks.Block, error)
}
```

Additional elements:

* The `go-blockservice.BlockGetter` interface should be replaced with the `Fetcher` interface.
* ~The output channel should have the same buffer as the input channel (I think?).~
* The `error` return type is for indicating that the request couldn't even be started (e.g., closed service). This interface doesn't really have a way to report runtime errors (and there's almost always nothing we can do about them anyways).

This will require changes to go-ipfs-exchange-interface, go-blockservice, and go-bitswap.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.