tensorflow / tensorflow/tensorflow

Description of Build Order and bootstrapped builds

Open
#53,582 21 comments 1 reaction 1 assignee View on GitHub

Nobody has claimed this yet.

override-stale stat:awaiting tensorflower type:build/install type:docs-feature
Dominant language
C++
Stars
200k
Forks
76.9k
Avg merge
2d 3h
Merged PRs (30d)
433

Description

I would like to request documentation on the ideal build order for tensorflow and both its required dependencies as well as suggested dependencies:

As described in: https://github.com/tensorflow/tensorflow/blob/v2.7.0/tensorflow/tools/pip_package/setup.py#L104
the package tensorflow depends on 4 dependencies that must be moved in step:

    'tensorboard ~= 2.6',
    'tensorflow_estimator ~= 2.7.0rc0, < 2.8',
    # Keras release is not backward compatible with old tf release, and we have
    # to make the version aligned between TF and Keras.
    'keras >= 2.7.0rc0, < 2.8',
    'tensorflow-io-gcs-filesystem >= 0.21.0',

However, it is my understanding that in order to build estimator that tensorflow must be importable by python.

Is that really the case?

If not, is there somewhere where we can read up on the ideal build order. Ideally we would have:

  1. Build tensorflow. This will create an importable package, but without many extra features.
    • It might be that this package is called tensorflow-base or something else to your liking
  2. Build estimator....
  3. Build keras ....
  4. Assemble all dependencies into a single dependency called tensorflow

Ideally this would form some directed acyclic graph so that we can bootstrap the builds without having an "older version of tensorflow" already compiled.

System information

  • OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Any
  • TensorFlow installed from (source or binary): source
  • TensorFlow version: 2.7.0
  • Python version: 3.7-3.10
  • Installed using virtualenv? pip? conda?: conda with conda-forge
  • Bazel version (if compiling from source): 4.XX
  • GCC/Compiler version (if compiling from source): 9
  • CUDA/cuDNN version: 11.2
  • GPU model and memory: Many

Describe the problem

It is best to look a the file recipe/build.sh that builds the wheel. Finally the file recipe/build_pkg.sh installs the wheel.
https://github.com/conda-forge/tensorflow-feedstock/pull/189

Any other info / logs
The best logs can be found on the conda-forge recipe. Here is the PR upgrading the system to 2.7.0. You can see a few patches strip out certain dependencies.

https://github.com/conda-forge/tensorflow-feedstock/pull/189

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.