oasisprotocol / oasisprotocol/oasis-core

Periodic 100% disk I/O utilization on Emerald running nodes.

Open
#4,577 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Go
Stars
369
Forks
151
Avg merge
1d 3h
Merged PRs (30d)
6

Description

SUMMARY

Experiencing 100% disk IO on node running Emerald paratime.

ISSUE TYPE
  • Bug Report
COMPONENT NAME

emerald paratime

OASIS NODE VERSION

Oasis core 21.3.9
emerald 7.1.0

OS / ENVIRONMENT
STEPS TO REPRODUCE
ACTUAL RESULTS

We are seeing periods of 100% disk I/O utilization on 2 nodes running Emerald paratime.
I/O utilization spikes up on both nodes at the same time.
Both nodes use NVME in RAID0 mode.

The same issue was seen on Emerald ver 6.*

Nodes running Cipher paratime do not exhibit the same behavior.

See screenshots below:

NODE 1:
Screen Shot 2022-03-18 at 00 12 52

NODE 2:
Screen Shot 2022-03-18 at 00 16 57

Please check if this might be causing transaction processing delays.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Begin by comparing the reported Emerald 7.1.0 behavior with Cipher nodes and the earlier Emerald 6.* behavior, using the two-node synchronized I/O spikes and NVMe RAID0 setup as the reproduction context. Determine whether transaction-processing delays correlate with the spikes; the issue names no source file or test, so code entry points and regression coverage still need to be located.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
blockchain, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.