[Bug-Candidate]: Poor scaling performance with high worker count - 80 workers 96 core cpu slower than 10 workers on a 12 core mac
- Dominant language
- Haskell
- Stars
- 3.2k
- Forks
- 432
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 5
Description
### Describe the issue:
## Issue Summary
Echidna appears to show limited performance gains when increasing the number of workers on high-core-count systems. In our tests, running with 80 workers on AMD EPYC 96 core hardware resulted in slower execution times compared to 10 workers on a Mac M2, despite the server offering significantly more cores and comparable single-thread performance.
## Performance Results
- **Local (Mac M2, 10 workers)**: 72.35 seconds real time, user time 8m59s
- **Remote (AMD EPYC, 80 workers)**: 104.12 seconds real time, , user time 90m18s
These results indicate that scaling is not linear, and may be affected by internal bottlenecks—potentially related to garbage collection, as noted in prior discussions with @gustavo-grieco
## Configuration
- testLimit: 10,000,000
- Contract: Simple dummy test contract
- Identical Echidna configuration across both environments, differing only in worker count
## Expected Behavior
With 8x more workers on similar single-core performance hardware, we anticipated faster execution on the AMD EPYC server, not 44% slower.
## Actual Behavior
Instead, increasing the worker count on high core-count CPUs appears to negatively impact performance. This may point to parallelization inefficiencies or Haskell runtime limitations when scaling across many cores.
## Environment
- Remote: AMD EPYC 9655 (96 cores) - 80 workers, 1.18 TB RAM
- Local: Apple M2 (12 cores) - 10 workers, 16gb RAM
- Echidna version: 2.2.6
### Code example to reproduce the issue:
```solidity
contract DummyTest {
uint256 public storedValue;
address public owner;
constructor() payable {
owner = msg.sender;
}
function setValues(uint256 _val, address _addr) public {
storedValue = _val;
owner = _addr;
}
}
```
// Echidna relevant config
```yaml
testLimit: 10000000
seqLen: 100
shrinkLimit: 1500
shrinkLimit: 1500
coverage: false
format: "text"
```
### Version:
2.2.6
### Relevant log output:
```shell
```
Contributor guide
Research direction
Reproduce the reported comparison using the Echidna 2.2.6 configuration, DummyTest contract, and worker counts described in the issue. No source file or test is named; trace the worker-scaling execution path and establish a measurable performance fix that avoids the reported slowdown at high worker counts.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- haskell, solidity
- Domain
- performance, testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100