tensorflow / tensorflow/build

Updating ppc64el wheel to latest version (currently: 2.2.0)

Open
#18 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Shell
Stars
311
Forks
139
Avg merge
1h 28m
Merged PRs (30d)
1

Description

Hi,

Expected Behavior

The ppc64el build at https://powerci.osuosl.org/job/TensorFlow2_PPC64LE_GPU_Release_Build/ should provide the latest stable version.

Actual Behavior

The version provided is 2.2.0, while the latest version is 2.3.1.

Details

I am using the ppc64el wheel linked above on a POWER8 system: it works fine, but it's still at 2.2.0.

The main issue I have is installation time, because it depends on a specific version of scipy. When installing the tensorflow wheel, pip needs to build this version of scipy, which takes about 20 minutes.
The dependency on scipy is not actually required and got removed in 2.3.1: https://github.com/tensorflow/tensorflow/pull/41867

So, a 2.3.1 ppc64el wheel would make installation significantly faster.

Building a newer wheel

I used the script https://github.com/tensorflow/build/blob/master/ci_environments/ppc64le/gpu_build.sh together with the ibmcom/tensorflow-ppc64le:gpu-devel-manylinux2014 docker image and I managed to build tensorflow 2.3.1 with it.

There was one issue with some versions of python in the container:

/opt/python/cp37-cp37/bin/python3.7: error while loading shared libraries: libcrypt.so.2: cannot open shared object file: No such file or directory

I could fix that by adding /usr/local/lib/ to $LD_LIBRARY_PATH (most notably in the .tf_configure.bazelrc file generated by the script).

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with ci_environments/ppc64le/gpu_build.sh and the ibmcom/tensorflow-ppc64le:gpu-devel-manylinux2014 image. Reproduce the TensorFlow 2.3.1 build, checking the libcrypt.so.2 failure and the LD_LIBRARY_PATH workaround described in the issue. Done means a current ppc64el GPU wheel is produced and no longer requires the older scipy version.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, python, shell, tensorflow
Domain
build-system, ci-cd
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.