tensorflow / tensorflow/build

Process for building Tensorflow that maximises use of system libraries?

Open
#42 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Shell
Stars
311
Forks
139
Avg merge
1h 28m
Merged PRs (30d)
1

Description

Hi there, I raised a question about this on the Discord server, but was advised to come here and ask instead.

Are there any instructions/guidelines on how to build a smaller, lighter Tensorflow shared library (C bindings) and also Python module that maximises the use of standard system libraries, eg in Ubuntu 18.04, 20.04, CentOS or MSYS2?

With 29 dependencies listed in the THIRD_PARTY_TC_C_LICENSES, it seems to me that this 320 MB shared library could be potentially much smaller if it made use of system libraries instead of statically linked versions, and if there is a decent TensorFlow test suite then it should not be too hard to assure that the library is still functioning correctly.

I note https://github.com/tensorflow/build/tree/master/official_build_environments ('They should have as few external dependencies as possible'). With the dockers and statically linked shared libraries, it seems that TensorFlow community has a strong bias against this approach, preferring more self-contained builds with less risk of platform-specific divergence. I just wonder if, as the code's user base grows, whether there might be a case for more 'standard' OS packaging of this library (eg .deb, .rpm) and then also the associated Python bindings, which presumably in the 'standard install' also contains a separate copy of the shared library?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the official_build_environments guidance and the THIRD_PARTY_TC_C_LICENSES reference mentioned in the issue. Compare the current Docker and statically linked build approach across the proposed Ubuntu, CentOS, and MSYS2 targets, then define what a supported system-library or OS-packaging process would need to document and test.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, python, tensorflow
Domain
build-system, release
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.