Multi stage in multiple files

Hi,

I am not sure if this is the right place to request for a change in the building and format of Dockerfiles.
Please redirect me if it is not the case.

It can be beneficial to projects to split a multi stage Dockerfile in several files.
For example, in a github repository pointing to another existing repository as a submodule that it cannot modify.
Alternately, in the context of a monorepo where several Dockerfiles exist for each component subfolder where one would want to reuse Dockerfile code. (DRY)

Today, as explained here when using multi stage strategies such as COPY --from= or a stage FROM one cannot point to a stage in another separate Dockerfile in a unique build command.

The closest way for a user to emulate this is to run several docker build commands with a crafted tag and use this tag in the subsequent Dockerfile.

There are several downsides to this, especially in the context of using an automatic build system:

• In some platforms it is not possible to modify the underlying build commands
• Some developers are not versed in crafting the build commands
• It would be non trivial to craft the docker commands in a way that respects multistage max caching

Would it be possible to introduce some syntax like follows that would, in a unique build command, execute the dockerfiles in succession and handle the tag as a classic stage:

(Mind that these dockerfiles are simplified for clarity purposes)

dev.Dockerfile

FROM python:latest as build-stage

# Create a virtual environment that will contain python libraries
RUN python -m venv /opt/venv  

# Install a library in the virtual environment
ENV PATH="/opt/venv/bin:$PATH"
RUN pip install numpy  

Dockerfile

FROM ./dev.Dockerfile:build-stage as test-stage

# Install a test library in the test-stage
RUN pip install pytest  

# ---

FROM python:slim as run-stage

ENV PATH=/opt/venv/bin:$PATH

# Copy the virtual environment to a leaner base image for a smaller runtime image on disk
COPY --from=./dev.Dockerfile:build-stage /opt/venv/. /opt/venv/

CMD ["python"]

Both dockerfiles would need to be in the build context, and paths would be relative to the build context.
The tag would simply be a stage name and every depending stage would be built on the fly with the usual pre-existing logic

Start by reviewing how a unique build command currently handles multi-stage FROM and COPY --from references, using the proposed dev.Dockerfile and Dockerfile examples as the target behavior. Check the linked Docker documentation for external-image stages and determine how cross-file stages in one build context could preserve normal dependency handling and caching. Done means dependent stages can be built from multiple Dockerfiles in one command without manually tagging intermediate images.