argotorg / argotorg/solidity

Contract compilation unusually slow on macOS

Open
#14,337 8 comments 0 reactions 0 assignees View on GitHub
bug :bug: performance :racehorse:
Dominant language
C++
Stars
25.7k
Forks
6.2k
Avg merge
1d 11h
Merged PRs (30d)
21

Description

I'm preparing a PR that will parallelize CLI tests (#14339) and in particular the via-ir equivalence will now get its own run. Comparing that between Ubuntu and macOS, I see that it runs 14x slower on the latter:
- [`~via_ir_equivalence` in `t_ubu_cli`](https://app.circleci.com/pipelines/github/ethereum/solidity/30231/workflows/df459314-b1a3-42d0-965a-4393aa90f727/jobs/1342181/parallel-runs/6?filterBy=ALL): 30 sec
- [`~via_ir_equivalence` in `t_osx_cli`](https://app.circleci.com/pipelines/github/ethereum/solidity/30231/workflows/df459314-b1a3-42d0-965a-4393aa90f727/jobs/1342203/parallel-runs/6?filterBy=ALL): 7 min

That's even bigger slowdown than we usually see when building solc, which is "only" 3x.

It also stands out compared to soltest, where macOS is about 2x slower:
- [`t_ubu_clang_soltest`](https://app.circleci.com/pipelines/github/ethereum/solidity/30231/workflows/df459314-b1a3-42d0-965a-4393aa90f727/jobs/1342189): ~8 min (20 parallel runs, ~25 sec each)
- [`t_osx_soltest`](https://app.circleci.com/pipelines/github/ethereum/solidity/30231/workflows/df459314-b1a3-42d0-965a-4393aa90f727/jobs/1342202): ~16 min

For via IR eqvivalence most of the execution time is taken by solc compiling contracts rather than evmone executing them or unrelated things like running clang. It's possible that it's not CI that's the culprit here but rather that solc is unusually slow on macOS.

The excessive slowdown does not seem to be caused exclusively by IR compilation. The legacy compilation tests are affected to a similar degree:
- [`~compilation_tests` in `t_ubu_cli`](https://app.circleci.com/pipelines/github/ethereum/solidity/30231/workflows/df459314-b1a3-42d0-965a-4393aa90f727/jobs/1342181/parallel-runs/3?filterBy=ALL): 30 sec
- [`~compilation_tests` in `t_osx_cli`](https://app.circleci.com/pipelines/github/ethereum/solidity/30231/workflows/df459314-b1a3-42d0-965a-4393aa90f727/jobs/1342203/parallel-runs/3?filterBy=ALL): 5 min

### Reproducing the problem
Here's how to run these particular tests locally:
```bash
test/cmdlineTests.sh ~via_ir_equivalence ~compilation_tests
```

Contributor guide

Open the contributing guide

Research direction

Start by running test/cmdlineTests.sh with ~via_ir_equivalence and ~compilation_tests on macOS and Ubuntu, then compare where solc spends time during compilation. Use the linked CircleCI runs as the baseline; done means identifying the source of the macOS slowdown and documenting or correcting it so the affected tests no longer show the excessive regression.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
compilers, performance, testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.