coinbase / coinbase/mesh-cli

Question about Performance: Processed block per second

Open
#316 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
165
Forks
99
PR merge metrics
No merged PRs in 30d

Description

Hi. I'm Denver.

**TLDR:**
* Is there any tips for maximizing performance(processing blocks more fast) when do `check:data`?

**Description**
I'm currently implementing rosetta-klaytn for Klaytn.
Actually rosetta-klaytn implementation has been completed (but not yet published to public), so I'm currently doing `check:data` test for it.

Klaytn's current block number is `91,145,672` which is nine times bigger than Ethereum.
Recently processed block number by check:data is `14,145,672` (There are still a lot of blocks left.)

The processing of `check:data` is 10 blocks per sec and I'm trying to maximize this performance by optimizing configuration values and fix some source codes.
Please give me some tips or advices if you know something.

**Machine SPEC I'm currently using**
AWS c5a.8xlarge
* 32 vCPU
* 64 GiB RAM

**Config I'm currently using**
```json
{
"network": {
"blockchain": "Klaytn",
"network": "Mainnet"
},
"online_url": "http://localhost:8080",
"data_directory": "cli-data-cypress",
"http_timeout": 120,
"max_retries": 15,
"max_online_connections": 256,
"max_sync_concurrency": 256,
"tip_delay": 300,
"max_reorg_depth": 100,
"log_configuration": false,
"compression_disabled":true,
"memory_limit_disabled":false,
"error_stack_trace_disabled": false,
"coin_supported": false,
"construction": null,
"data": {
"active_reconciliation_concurrency": 16,
"inactive_reconciliation_concurrency": 32,
"inactive_reconciliation_frequency": 250,
"log_blocks": true,
"log_transactions": true,
"log_balance_changes": true,
"log_reconciliations": true,
"ignore_reconciliation_error": false,
"reconciliation_disabled": false,
"reconciliation_drain_disabled": false,
"inactive_discrepancy_search_disabled": false,
"balance_tracking_disabled": false,
"coin_tracking_disabled": false,
"status_port": 8081,
"results_output_file": "",
"pruning_disabled": false,
"initial_balance_fetch_disabled": false,
"historical_balance_disabled": false,
"bootstrap_balances": "bootstrap_balances.json"
}
}
```

And I changed [DefaultCacheSize](https://github.com/coinbase/rosetta-sdk-go/blob/8fa4542a6e04055c63979afa44b55afed7d8682f/syncer/types.go#L36-L38) from `2000 << 20` to `8000 << 20` to increase cache size to get much blocks from Block Fetcher.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.