internetarchive / internetarchive/openlibrary

Updates to node_modules on image rebuild not reflected in container's node_modules volume

Open
#2,032 2 comments 1 reaction 0 assignees View on GitHub
Affects: Configuration Lead: @cdrini Module: Docker Priority: 3 Theme: Development Type: Refactor/Clean-up
Dominant language
Python
Stars
6.7k
Forks
2k
Avg merge
2d 19h
Merged PRs (30d)
138

Description

## Description
Because we volume mount `./` to `/openlibrary`, `/openlibrary/node_modules` (which is populated when `npm install` is run inside the `Dockerfile`) gets rewritten by what's in the local directory. We've solved this issue by adding another volume mount specifically to `/openlibrary/node_modules`. This has had the unintended consequence that if the images are rebuilt, even though `npm install` is run there, when they are launched, they attach to the volume which contains the _old_ version of `/openlibrary/node_modules`. So `npm install` has to be run again to update the volume.

### What we want:
1. Rebuilding the image should run `npm install`
2. Running `docker-compose up` should **NOT** require running `npm install`
3. Running `docker-compose up` should have the same npm versions from the image.

## External links:
- https://jdlm.info/articles/2016/03/06/lessons-building-node-app-docker.html#the-node_modules-volume-trick : a guide on how this problem was handled in 2016 on another project. They use the same kind of volume mount trick, but using non-named / non-persistent volumes. I will investigate whether moving back to non-named volumes will guarantee they are created fresh every `docker-compose up` with the correct and latest `node_modules` from the base image.
The article has a very good description of the node_modules problem in the section "The node_modules Volume Trick" and how mounting the local dev dir overwrites the installed image modules with the empty host `node_modules`. The blue / red filenames is a great way to illustrate a problem (I have found) difficult to describe clearly.

Contributor guide

Open the contributing guide

Research direction

Start by inspecting the Dockerfile and docker-compose configuration, focusing on how npm install populates node_modules and how the additional volume is mounted. Rebuild the image and run docker-compose up to compare the container's npm versions with those in the image. Done means rebuilding updates the image modules and docker-compose up needs no manual npm install.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, node.js
Domain
devops, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.