chipsalliance / chipsalliance/Surelog

Antlr4 warm-up times - can this be improved/pre-computed ?

Open
#117 9 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
475
Forks
90
Avg merge
1h 39m
Merged PRs (30d)
37

Description

We'd like to add Surelog to [sv-tests](https://github.com/SymbiFlow/sv-tests), see this https://github.com/SymbiFlow/sv-tests/pull/447 pull request.

However, currently the tests are using more time that can be allocated by the continuous integration sysstem, so it is not merged yet. It seems the overall time for Surelog is dominated by start-up cost: it typically takes about 500 milliseconds (possibly due to the Antlr4 startup-time ?), while other binaries usually have 10ms start-up time.

### Reproduce

To verify this, it is possible to get the merge request locally and run sv-tests with Surelog
```
git clone https://github.com/SymbiFlow/sv-tests.git

cd sv-tests
git fetch origin pull/447/head:surelog-test
git checkout surelog-test
git submodule update --init --recursive

# this submodule-init might require pressing return a couple of times as there is
# some glitch in some submodule (basejump) that can't be retrieved.
# It asks for username/password, but just pressing return is ok.
```

Now, optionally, install the build-system `bazel` if comparison with [verible](https://github.com/google/verible) is desired
https://docs.bazel.build/versions/1.2.0/install-ubuntu.html

```
# Build the tools we're interested in.
make -j10 -k surelog yosys verible

# Now run the tests and generate the reports
make generate-tests
make -j10 -k tests
make report
```
The results appear in `out/report/index.html` ; the total runtime of Yosys or Verible for all tests combined is in the order of 30 seconds while with all the start-up costs it is > 1000 seconds for Surelog.

### Smaller tests are dominated by start-up time
Each of the small tests is dominated by the start-up time. Is there a way to have the Antlr4 'warm-up' status pre-generated and compiled into the system to minimize this ?

### Finding performance optimization opportunities

The run-time is also recorded in the log-files directly, so it is possible to inspect these to look for low-hanging fruit of possible improvements:
```
find out/logs/Surelog/ -name "*.log" | xargs grep time_elapsed | awk '{printf("%8.3f %s\n", $2, $1); T+=$2} END{ printf("%8.3f Total time\n", T);}' | sort -n
```

The sv-tests also help to identify potentially low-hanging fruit of larger compilation times.

Contributor guide

No contributing guide indexed for this repository

Research direction

Reproduce the startup-cost report by checking out sv-tests pull/447 and running its make commands for Surelog, then inspect out/logs/Surelog/*.log and the generated report. Compare the per-test and total timings with Yosys and Verible; done means Surelog's measured runtime is reduced enough to address the reported CI-time problem.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
compilers, performance
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.