chipsalliance / chipsalliance/Surelog
Antlr4 warm-up times - can this be improved/pre-computed ?
- Dominant language
- C++
- Stars
- 475
- Forks
- 90
- Avg merge
- 1h 39m
- Merged PRs (30d)
- 37
Description
We'd like to add Surelog to [sv-tests](https://github.com/SymbiFlow/sv-tests), see this https://github.com/SymbiFlow/sv-tests/pull/447 pull request.
However, currently the tests are using more time that can be allocated by the continuous integration sysstem, so it is not merged yet. It seems the overall time for Surelog is dominated by start-up cost: it typically takes about 500 milliseconds (possibly due to the Antlr4 startup-time ?), while other binaries usually have 10ms start-up time.
### Reproduce
To verify this, it is possible to get the merge request locally and run sv-tests with Surelog
```
git clone https://github.com/SymbiFlow/sv-tests.git
cd sv-tests
git fetch origin pull/447/head:surelog-test
git checkout surelog-test
git submodule update --init --recursive
# this submodule-init might require pressing return a couple of times as there is
# some glitch in some submodule (basejump) that can't be retrieved.
# It asks for username/password, but just pressing return is ok.
```
Now, optionally, install the build-system `bazel` if comparison with [verible](https://github.com/google/verible) is desired
https://docs.bazel.build/versions/1.2.0/install-ubuntu.html
```
# Build the tools we're interested in.
make -j10 -k surelog yosys verible
# Now run the tests and generate the reports
make generate-tests
make -j10 -k tests
make report
```
The results appear in `out/report/index.html` ; the total runtime of Yosys or Verible for all tests combined is in the order of 30 seconds while with all the start-up costs it is > 1000 seconds for Surelog.
### Smaller tests are dominated by start-up time
Each of the small tests is dominated by the start-up time. Is there a way to have the Antlr4 'warm-up' status pre-generated and compiled into the system to minimize this ?
### Finding performance optimization opportunities
The run-time is also recorded in the log-files directly, so it is possible to inspect these to look for low-hanging fruit of possible improvements:
```
find out/logs/Surelog/ -name "*.log" | xargs grep time_elapsed | awk '{printf("%8.3f %s\n", $2, $1); T+=$2} END{ printf("%8.3f Total time\n", T);}' | sort -n
```
The sv-tests also help to identify potentially low-hanging fruit of larger compilation times.
Contributor guide
No contributing guide indexed for this repository
Research direction
Reproduce the startup-cost report by checking out sv-tests pull/447 and running its make commands for Surelog, then inspect out/logs/Surelog/*.log and the generated report. Compare the per-test and total timings with Yosys and Verible; done means Surelog's measured runtime is reduced enough to address the reported CI-time problem.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- compilers, performance
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100