Container Road Map

Road Map for Docker Containers

This is the same roadmap document that I'm using internally, with the internal bits taken out.

I am forcing these containers to get continuous support by using them for TF's internal CI: if they don't work, then our tests don't work. While I'm getting that ready during Q4 and Q1, I'm explicitly avoiding features that the TF team is not going to use, which would be dead-on-arrival unless we set up more testing for them, which I don't have the cycles to consider yet.

TF Nightly Milestone - Q4 Q1

Goal: Replicable container build of our tf-nightly Ubuntu packages

• Containers can build tf-nightly package
• SIG Build repository explains how to build tf-nightly package in Containers
• Documentation exists on how to make changes to the containers
• Suite of Kokoro jobs exists that publishes 80%-identical-to-now tf-nightly via containers
• TF-nightly is officially built with the new containers
• Documentation exists on how to use and debug containers

Release Test Milestone - Q4 Q1

Goal: Replicable container builds of our release tests, supporting each release

• Containers can run same-as-now Nightly release tests
• SIG Build repository explains how to run release tests as we do
• Suite of CI jobs exists that matches current rel/nightly jobs
• Existing release jobs replaced (but reversible if needed) by Container-based equivalent
• Containers may be maintained and updated separately for TF release branches
• Containers used for nightly/release libtensorflow and ci_sanity (now "code_check") jobs

CI & RBE Milestone - Q4 Q1/Q2

Goal: The main tests and our RBE tests use the same Docker container, updated in one place

• Containers support internal presubmit/continuous build behavior
• Containers are used in internal buildcop-monitored, DevInfra-owned presubmit/continuous jobs
• Containers can be used in RBE
• Containers are used as RBE environment for internal buildcop-monitored, DevInfra-owned jobs
• DevInfra's GitHub-side presubmit tests use the containers
• Containers are published on gcr.io
• There is an easy way to verify if a change to the containers will not break the whole internal test suite

Forward Planning Milestone - Q2

Goal: Establish clear plan for any future work related to these containers. This is internal team planning stuff so I've removed it.

Downstream & OSS Milestone - Q2/Q3

Goal: Downstream users and custom-op developers use the same containers as our CI

• SIG Addons / SIG IO use these Containers (or derivative) instead of old custom-op ones
• Custom-op documentation migrated to SIG Build repository
• Resolve: what to do about inconvenient default packages for e.g. SIG Addons (keras-nightly, etc.)
• Resolve: what to do about inconveniently large image sizes for e.g. GPU content not needed
• Docker-related documentation on tensorflow.org replaced with these containers
• "devel" containers deprecated in favor of SIG Build containers

This issue is a historical roadmap rather than an actionable change, and it names no file, test, or entry point. Start by reviewing the unchecked Downstream & OSS Milestone items and the repository's current container documentation; a concrete contribution would need one selected goal with a defined completion criterion.