onflow / onflow/flow

FLIP [Draft]: FifoQueue for engines

Open
#396 10 comments 0 reactions 1 assignee View on GitHub

@AlexHentschel is already working on this.

Since Feb 22, 2021.

FLIP
Dominant language
Go
Stars
361
Forks
159
PR merge metrics
No merged PRs in 30d

Description

we decided to introduce unblocking message queue(s) for the engines, so that they don’t block the networking layer’s queue and thereby each other. Yurii has already made great progress on the consensus nodes implementing such queues. I recently found a blog post, where a similar implementation is discussed. Comparing this proposal to our implementation (e.g. here), I notice two noteworthy differences

  • I really like our queue FifoQueue implementation (essentially wrapper around deque.Deque) much better than using slices
  • In our implementation, we also have an inbound and outbound channel, with some internal unbounded storage slice in the middle. However, the go routines which shovel the data from the inbound channel -> internal storage -> outbound channel live in the engine (e.g. here). Thereby, we replicate this code in every engine.

I was was wondering, what the thoughts where on moving this go-routine for shovelling the queue elements into the FifoQueue. From my perspective, this would generate a very clean separation of concerns. I think we could take the shovelling logic Jon Bodner's blog post with only minimal changes.

This is mainly a question for @durkmurder, @zhangchiqing, and @arrivets but I thought I open it up for a broader discussion here.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.