algolia / algolia/docsearch-scraper

Optimize docker image

Open
#459 7 comments 1 reaction 0 assignees View on GitHub
bug enhancement help wanted
Dominant language
Python
Stars
322
Forks
115
PR merge metrics
No merged PRs in 30d

Description

The docker image that's being published for this repository is severely fragmented. As a result, it takes a much longer time to download than it should and consumes a lot of extra disk space. I count nearly 30 layers, which occupies close to 2GB. It really should be just one layer. Please optimize the image by consolidating the RUN commands or by using the `--squash` flag when building.

Also, what's going on with the versions? Here are the versions I see on [DockerHub](https://hub.docker.com/r/algolia/docsearch-scraper/tags):

```
v0.9
v1.1
v1.1.0
v1.2.0
v1.3
v1.3.1
```

Can we stick to a consistent pattern? And how was v0.9 published after v1.2.0?

Even more concerning, v1.3.1 is absolutely not a patch release. It changed the version of Python from 2 to 3 and added 200MB to the size of the image. Can we follow semantic versioning?

I expect better software practices from Algolia than this. Let's work towards that.

Contributor guide

Open the contributing guide

Research direction

Start by inspecting the Docker build configuration and the published DockerHub tags for algolia/docsearch-scraper. Compare the image layers, size, Python version, and tag history; done means a smaller consolidated image and a consistent versioning scheme that reflects the changes.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, python
Domain
devops, release
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.