apache / apache/openwhisk

Analyze OpenWhisk Scala compilation for performance

Open
#4,509 8 comments 4 reactions 0 assignees View on GitHub
Dominant language
Scala
Stars
6.8k
Forks
1.2k
Avg merge
2d 14h
Merged PRs (30d)
2

Description

Currently the Scala compilation phase for OpenWhisk take ~ 2-2.5 mins for full compilation. Recently I have been trying out the [Hydra Compiler][1] along with its [monitoring][2] to gain insights into how our build is performing.

Purpose of this issue is to better understand the build and see if we can reduce the compilation time further. May be we can boost the default performance [quite a bit][4]

See [Branch with hydra enabled][3] on changes required in build. [Dashboard doc][5] explains the details of various stats

### Scala Common Module

![image](https://user-images.githubusercontent.com/664531/59031283-dc04bc00-8880-11e9-9d75-d503358f3fa4.png)

Keys points

* Takes ~ 30 secs for clean build with Hydra boost enabled
* MesosContainerFactory takes ~ 9 secs
* if we can optimize the compilation for top5 we can possibly reduce the time taken to half say 10-15 secs

Other modules

* Invoker takes 18 secs
* Controller takes 7 sec
* Test take 25 secs

[1]: https://triplequote.com/hydra
[2]: https://triplequote.com/hydra/monitoring/
[3]: https://github.com/apache/incubator-openwhisk/compare/master...chetanmeh:hydra
[4]: https://jobs.zalando.com/tech/blog/achieving-3.2x-faster-scala-compile-time/
[5]: https://docs.triplequote.com/dashboard/user/

Contributor guide

Open the contributing guide

Research direction

Start with the Hydra-enabled comparison branch and the Hydra monitoring/dashboard documentation, then compare clean-build timings for Scala Common, Invoker, Controller, and tests. Focus first on the reported MesosContainerFactory cost and top compilation contributors; done means identifying actionable build changes and demonstrating reduced compilation time with measurements.

Written by the indexing model from the issue text.

Assessment

Tech stack
scala
Domain
build-system, performance
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.