Add arm64 third-party CI
Nobody has claimed this yet.
- Dominant language
- Shell
- Stars
- 311
- Forks
- 139
- Avg merge
- 1h 28m
- Merged PRs (30d)
- 1
Description
System information
TensorFlow version (you are using): 2.0+ and master branch
Are you willing to contribute it (Yes/No): Yes
Describe the feature and the current behavior/state.
Currently, Tensorflow only has the official build CI on X86 and third-party build CIs on x86 and ppc64. There is no CI for arm64. Adding a arm64 CI can help community to discover arm64 problems easily.
OpenLab supports public CI system for opensource projects[1]. Now it supports arm64 arch and the tensorflow nightly build jobs for 2.0+ and master version have been added there as well[2]. It runs tensorflow build everyday at UTC-18.
Just like what tensorflow do currently, we can just easily add a new badge in the README.md file to link the Openlab arm64 third-party CI.
As you can see in the page[2], tensorflow 2.0, 2.1 and 2.2(master) build well on aarch64. But in some aws libs strongly based on x86 ARCH, so in master branch, I skip that part for build. You can see the build brief in [3], download the build whl packages there, and see the details logs in [4].
So adding the arm64 build CI is useful for the community.
1: https://openlabtesting.org
2: http://status.openlabtesting.org/builds?project=tensorflow%2Ftensorflow
3: http://status.openlabtesting.org/build/c816e5c9d6cc4519b933414fc6044d28
4: https://logs.openlabtesting.org/logs/periodic-18/github.com/tensorflow/tensorflow/master/tensorflow-arm64-build-daily-v2.1.0/c816e5c/
Additional context
Now the test is CPU only and is basing on Ubuntu 18.04 and python3.6. More can be added in the future.
And I'm from OpenLab commuinty. I'll keep looking after the tensorflow arm64 CI and try my best to fix the arm64 failure then.
Here is an example[1] we done in pytorch community. See 'Linux (aarch64) CPU' badge in the README.md.
Also for another community[2], we done in greenplum-db community. See 'Zuul Regression Test On Arm' badge in the README.md
1: https://github.com/pytorch/pytorch/blob/master/README.md
2: https://github.com/greenplum-db/gpdb
Will this change the current api? How?
No
Who will benefit with this feature?
The arm64 users and developers
- Notes
And there is some other discussion about the same problem in tensorflow repo. see https://github.com/tensorflow/tensorflow/issues/40463
https://groups.google.com/a/tensorflow.org/forum/#!topic/build/zTbmc0T6jAw
And for now, @AshokBhat is working on fix the aws-lab libs for ARM support. We already build the master branch after https://github.com/tensorflow/tensorflow/pull/40700 in my local repo, and downgrade numpy via pip3 install numpy==1.18.0 with cmd:
bazel clean --expunge ; bazel build --config=opt //tensorflow/tools/pip_package:build_pip_package --local_ram_resources=10240 --local_cpu_resources=7 --verbose_failures
All thanks you guys and @AshokBhat 's kind help.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with README.md and inspect the existing CI badges and their link formats. Use the OpenLab TensorFlow build page and linked build details to confirm the arm64 job and badge target, then add the arm64 third-party CI badge and verify that it points to the published build status.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, python, ubuntu
- Domain
- ci-cd, documentation
- Issue type
- Feature
- Difficulty
- 1/5
- Estimated time
- Under an hour
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100