OpenAPITools / OpenAPITools/openapi-generator

[BUG] [python-fastapi] Improved DockerFile caching with pip install

Open
#17,833 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Issue: Bug
Dominant language
Java
Stars
26.8k
Forks
7.7k
PR merge metrics
PR metrics pending

Description

Bug Report Checklist
  • Have you provided a full/minimal spec to reproduce the issue?
  • Have you validated the input using an OpenAPI validator (example)?
  • Have you tested with the latest master to confirm the issue still exists?
  • Have you searched for related issues/PRs?
  • What's the actual output vs expected output?
  • [Optional] Sponsorship to speed up the bug fix or feature request (example)

Description

The generated DockerFile re-installs the python dependencies on every change of the underlying code instead of performing that separately from installing the generated library

openapi-generator version

the online test generator
(or the master one)

OpenAPI declaration file content or url

The petstore API

Generation Details

Steps to reproduce

  • Generate the server stubs with the python-fastapi generator
  • run docker compose up --build in the generated code directory (as directed by the generated README.md)
  • change/add/remove/etc. a file in the project
  • re-run docker compose up --build again
  • The command takes a long time as the cached pip install is dependent on the invalidated (changed) source files and re-installs all of the dependencies from scratch again.

Related issues/PRs

A fixing PR could also address the DockerFile bug reported in #11006. (Copy from builder instead of test_runner)

Suggest a fix

Add

COPY ./requirements.txt ./requirements.txt
RUN pip install --no-cache-dir -r ./requirements.txt

right before

COPY . .
RUN pip install --no-cache-dir .

Unfortunately, there is one issue with doing this, which is that in the setup.cfg there are additional dependencies specified via

[options]
install_requires = fastapi[all]

that don't get respected when breaking the install into two steps as suggested. This should also get moved into the requirements.txt.

However, changing the line in requirements.txt specifying the fastapi==... dependency to fastapi[all]==... introduces dependency version issues. Thus, there is some work to re-adjust the pinned versions of the dependencies to be compatible.
Note that for myself I adjusted all of the version pinning from a strict == pin to a looser >= and let pip sort things out and have not noticed any issues so far. So doing that could be a workaround until versions get sorted, but YMMV.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the generated Dockerfile, requirements.txt, and setup.cfg for the python-fastapi petstore output, then reproduce the problem with docker compose up --build from the generated README.md. Check how separating dependency installation affects setup.cfg's fastapi[all] requirement and pinned versions. Done means subsequent source changes reuse the dependency layer without breaking the generated install.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, docker-compose, fastapi, python
Domain
backend, build-system, devops
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.