oasisprotocol / oasisprotocol/oasis-core

go: Storage committee worker should be responsible for both runtime light history and state

Open
#6,400 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Go
Stars
369
Forks
151
Avg merge
1d 3h
Merged PRs (30d)
6

Description

As pointed out in the mini design doc, go/worker/storage/committee should be responsible for both updating and pruning of both runtime state and light history.

As such, this worker should also provide a synchronized view of the two databases to any client that requires it. This would also resolve the current issue of history.History containing synchronization logic it shouldn’t be responsible for.

Currently, storage committee worker is only started in case of the local storage (see). However, even without the runtime state, you may still have light history locally, and there is nothing stateless about it.

Instead this worker should handle two levels of local storage/state:

  1. Light storage only (runtime history).
  2. Full storage in combination with 1.
    • Significantly overlaps with #6307.

In the future this enables us to:

  1. Create seperate p2p protocol for runtime light headers, so that stateless clients do not reindex history (using stateless client backend - slow), but instead fetches it from the p2p network.
  2. Implement different pruning strategies for consensus/runtime light/full state.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the linked mini design doc, go/worker/storage/worker.go, and the storage committee worker and history.History it references. Compare the scope with #6307; done means the worker owns updating and pruning light history and runtime state, provides their synchronized view, and is started for light storage as well as full storage.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
databases, distributed-systems
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.