Building is very slow because of Python 2 and `ulimit` of number of open files
- Dominant language
- Scala
- Stars
- 6.8k
- Forks
- 1.2k
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 2
Description
## Environment details:
(all that matters is that I run on ArchLinux)
* host: ArchLinux (2023-04-20), Linux 6.2.11
* Gradlew:
```
------------------------------------------------------------
Gradle 6.9.1
------------------------------------------------------------
Build time: 2021-08-20 11:15:18 UTC
Revision: f0ddb54aaae0e44f0a7209c3c0274d506ea742a0
Kotlin: 1.4.20
Groovy: 2.5.12
Ant: Apache Ant(TM) version 1.10.9 compiled on September 27 2020
JVM: 1.8.0_362 (Oracle Corporation 25.362-b09)
OS: Linux 6.2.11-arch1-1 amd64
```
## Steps to reproduce the issue:
1. clone OpenWhisk source repository
2. `./gradlew distDocker`
## Provide the expected results and outputs:
OpenWhisk should build in reasonable time (a matter of minutes, less than 3min on my computer).
## Provide the actual results and outputs:
OpenWhisk is very slow to build.
The very slow phase is building ow-utils, in particular installing Python 2 native modules which also compiles them (>=30min and counting, I stopped after this time).
## Proposed fix
Move to Python 3-only base Docker images.
Workaround: set a `ulimit` directive on Docker build commands in "gradle/docker.gradle" to lower the limit on the number of open files (`nofile`).
For example:
```gradle
dockerBuildArg = ['build', '--ulimit', 'nofile=4096:4096']
```
## Additional information you deem important:
The underlying problem has been identified in an issue about another topic in [openwhisk-deploy-kube](https://github.com/apache/openwhisk-deploy-kube/issues/673#issuecomment-869067074): Python 2 does not handle well high numbers of possibly open FDs when asked to close them when calling `subprocess.Popen` (see [Python bug report](https://bugs.python.org/issue1663329)).
What this has to do with building OpenWhisk, is that the Docker image for ow-utils (possibly others) uses Python 2 and installs Python packages with native modules; the latter must be compiled on-the-fly, incurring a lot of calls to `subprocesss.Popen` (just a guess here) that trigger this behavior.
It is very visible on ArchLinux because the default ulimit on the number of open files in a Docker container is very high in this distribution (not sure why not in others, I guess they set a lower default value system-wide).
Honestly, I think effort should be made to definitely migrate from Python 2 in every OpenWhisk component.
Contributor guide
Assessment
This issue has not been assessed yet.