Analyze OpenWhisk Scala compilation for performance
- Dominant language
- Scala
- Stars
- 6.8k
- Forks
- 1.2k
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 2
Description
Currently the Scala compilation phase for OpenWhisk take ~ 2-2.5 mins for full compilation. Recently I have been trying out the [Hydra Compiler][1] along with its [monitoring][2] to gain insights into how our build is performing.
Purpose of this issue is to better understand the build and see if we can reduce the compilation time further. May be we can boost the default performance [quite a bit][4]
See [Branch with hydra enabled][3] on changes required in build. [Dashboard doc][5] explains the details of various stats
### Scala Common Module

Keys points
* Takes ~ 30 secs for clean build with Hydra boost enabled
* MesosContainerFactory takes ~ 9 secs
* if we can optimize the compilation for top5 we can possibly reduce the time taken to half say 10-15 secs
Other modules
* Invoker takes 18 secs
* Controller takes 7 sec
* Test take 25 secs
[1]: https://triplequote.com/hydra
[2]: https://triplequote.com/hydra/monitoring/
[3]: https://github.com/apache/incubator-openwhisk/compare/master...chetanmeh:hydra
[4]: https://jobs.zalando.com/tech/blog/achieving-3.2x-faster-scala-compile-time/
[5]: https://docs.triplequote.com/dashboard/user/
Contributor guide
Research direction
Start with the Hydra-enabled comparison branch and the Hydra monitoring/dashboard documentation, then compare clean-build timings for Scala Common, Invoker, Controller, and tests. Focus first on the reported MesosContainerFactory cost and top compilation contributors; done means identifying actionable build changes and demonstrating reduced compilation time with measurements.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- scala
- Domain
- build-system, performance
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100