aboutcode-org / aboutcode-org/scancode-toolkit

Optimize Dockerfile layer caching to improve build performance

Open
#4,632 1 comment 0 reactions 0 assignees View on GitHub
new feature
Dominant language
Python
Stars
2.6k
Forks
791
Avg merge
1d 12h
Merged PRs (30d)
5

Description

## Short Description

Restructure Dockerfile COPY commands to improve layer caching efficiency and reduce Docker build times.

## Possible Labels

- enhancement
- docker

## Select Category

- [x] Enhancement
- [ ] Add License/Copyright
- [ ] Scan Feature
- [x] Packaging
- [ ] Documentation
- [ ] Expand Support
- [ ] Other

## **Describe the Update**

The current Dockerfile uses `COPY . /scancode-toolkit` which copies the entire source directory before running the configuration step. This approach invalidates all subsequent Docker layers whenever any file changes (including documentation, samples, or tests), forcing unnecessary rebuilds of the expensive `./configure` and license indexing steps.

This update restructures the COPY commands to:
1. Copy only essential configuration and source files first
2. Run the configuration and license indexing steps
3. Copy remaining runtime files afterwards

## **How This Feature will help you/your organization**

**Benefits:**
- **Faster Docker builds**: Changes to non-essential files (docs, samples, README) won't invalidate the configure layer
- **Improved CI/CD performance**: Significantly reduced build times in continuous integration pipelines
- **Better developer experience**: Local Docker builds complete faster during development
- **Cost savings**: Reduced compute time in cloud-based CI/CD systems

**Impact:** The `./configure` and `scancode-reindex-licenses` steps are time-consuming. With proper layer caching, these only rebuild when actual source code or configuration files change, not when documentation is updated.

## **Possible Solution/Implementation Details**

Replace the single `COPY . /scancode-toolkit` command with selective, ordered COPY commands:

```dockerfile
# Copy configuration files first
COPY configure configure.bat setup.py setup.cfg setup-mini.cfg pyproject.toml MANIFEST.in ./
COPY requirements*.txt ./
COPY src/ ./src/
COPY etc/ ./etc/

# Run configuration (this layer is now cached unless above files change)
RUN ./configure && ./venv/bin/scancode-reindex-licenses

# Copy runtime files
COPY scancode scancode.bat extractcode extractcode.bat ./
COPY *.rst *.LICENSE *.ABOUT NOTICE ./

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.