aboutcode-org / aboutcode-org/scancode-toolkit
Optimize Dockerfile layer caching to improve build performance
- Dominant language
- Python
- Stars
- 2.6k
- Forks
- 791
- Avg merge
- 1d 12h
- Merged PRs (30d)
- 5
Description
## Short Description
Restructure Dockerfile COPY commands to improve layer caching efficiency and reduce Docker build times.
## Possible Labels
- enhancement
- docker
## Select Category
- [x] Enhancement
- [ ] Add License/Copyright
- [ ] Scan Feature
- [x] Packaging
- [ ] Documentation
- [ ] Expand Support
- [ ] Other
## **Describe the Update**
The current Dockerfile uses `COPY . /scancode-toolkit` which copies the entire source directory before running the configuration step. This approach invalidates all subsequent Docker layers whenever any file changes (including documentation, samples, or tests), forcing unnecessary rebuilds of the expensive `./configure` and license indexing steps.
This update restructures the COPY commands to:
1. Copy only essential configuration and source files first
2. Run the configuration and license indexing steps
3. Copy remaining runtime files afterwards
## **How This Feature will help you/your organization**
**Benefits:**
- **Faster Docker builds**: Changes to non-essential files (docs, samples, README) won't invalidate the configure layer
- **Improved CI/CD performance**: Significantly reduced build times in continuous integration pipelines
- **Better developer experience**: Local Docker builds complete faster during development
- **Cost savings**: Reduced compute time in cloud-based CI/CD systems
**Impact:** The `./configure` and `scancode-reindex-licenses` steps are time-consuming. With proper layer caching, these only rebuild when actual source code or configuration files change, not when documentation is updated.
## **Possible Solution/Implementation Details**
Replace the single `COPY . /scancode-toolkit` command with selective, ordered COPY commands:
```dockerfile
# Copy configuration files first
COPY configure configure.bat setup.py setup.cfg setup-mini.cfg pyproject.toml MANIFEST.in ./
COPY requirements*.txt ./
COPY src/ ./src/
COPY etc/ ./etc/
# Run configuration (this layer is now cached unless above files change)
RUN ./configure && ./venv/bin/scancode-reindex-licenses
# Copy runtime files
COPY scancode scancode.bat extractcode extractcode.bat ./
COPY *.rst *.LICENSE *.ABOUT NOTICE ./
Contributor guide
Assessment
This issue has not been assessed yet.