AlexsLemonade / AlexsLemonade/refinebio
Docker cache miss during deploy
- Dominant language
- Python
- Stars
- 135
- Forks
- 21
- PR merge metrics
- No merged PRs in 30d
Description
### Context
During deploy process _some_ docker images still being rebuilt from scratch.
### Problem or idea
Smasher:
```
#9 importing cache manifest from ccdlstaging/dr_smasher_cache:latest
#9 DONE 0.3s
#6 [ 1/12] FROM docker.io/ccdlstaging/dr_base:v1.45.7-dev@sha256:776a8c605a6a04914572f59d0be4989b8c9fe05011d62466bd167af90868c2f3
#6 DONE 0.0s
#5 [internal] load build context
#5 transferring context: 1.56MB 0.1s done
#5 DONE 0.1s
#10 [ 2/12] WORKDIR /home/user
#10 CACHED
#11 [ 3/12] COPY workers/R/dependencies/smasher/renv.lock .
#11 DONE 0.1s
#12 [ 4/12] COPY common/R/renv_load.R .
#12 DONE 0.0s
#13 [ 5/12] RUN Rscript renv_load.R
#13 0.463 Installing package into ‘/usr/local/lib/R/site-library’
#13 0.463 (as ‘lib’ is unspecified)
#13 2.414 trying URL 'https://bioconductor.org/packages/3.6/bioc/src/contrib/BiocInstaller_1.28.0.tar.gz'
#13 2.709 Content type 'application/octet-stream' length 18624 bytes (18 KB)
#13 2.722 ==================================================
```
Salmon:
```
#9 importing cache manifest from ccdlstaging/dr_salmon_cache:latest
#9 DONE 0.2s
#6 [ 1/14] FROM docker.io/ccdlstaging/dr_base:v1.45.7-dev@sha256:776a8c605a6a04914572f59d0be4989b8c9fe05011d62466bd167af90868c2f3
#6 DONE 0.0s
#5 [internal] load build context
#5 transferring context: 14.06kB 0.1s done
#5 DONE 0.1s
#10 [ 2/14] WORKDIR /home/user
#10 CACHED
#11 [ 3/14] COPY workers/R/dependencies/salmon/renv.lock .
#11 CACHED
#12 [ 4/14] COPY common/R/renv_load.R .
#12 DONE 0.1s
#13 [ 5/14] RUN Rscript renv_load.R
#13 0.358 Installing package into ‘/usr/local/lib/R/site-library’
#13 0.358 (as ‘lib’ is unspecified)
#13 2.507 trying URL 'https://bioconductor.org/packages/3.6/bioc/src/contrib/BiocInstaller_1.28.0.tar.gz'
#13 2.803 Content type 'application/octet-stream' length 18624 bytes (18 KB)
#13 2.821 ==================================================
```
### Solution or next step
Figure out the layer cache invalidation reason and fix it.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by comparing the Docker build logs for Smasher and Salmon, especially the cache manifest import and the COPY steps for workers/R/dependencies/*/renv.lock and common/R/renv_load.R. Trace why the RUN Rscript renv_load.R layer is rebuilt during deploy. Done means the layer cache is reused when inputs have not changed and the affected images no longer rebuild dependencies from scratch.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, r
- Domain
- build-system, devops
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100