make jemalloc: download failure is silently swallowed and never retried, surfacing as a misleading tar error
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 21.8k
- Forks
- 1.6k
- Avg merge
- 2d 5h
- Merged PRs (30d)
- 9
Description
Summary
The jemalloc target in dgraph/Makefile chains its download and extraction with ; rather than &&, so a failed curl does not stop the recipe. The build then fails several steps later with a misleading error, and the real cause — the download — is discarded.
There is also no retry on what is a network fetch from a GitHub release, so a single transient blip fails the build.
The code
dgraph/Makefile, jemalloc target:
mkdir -p /tmp/jemalloc-temp && cd /tmp/jemalloc-temp ; \
echo "Downloading jemalloc" ; \
curl -f -s -L ${JEMALLOC_URL} -o jemalloc.tar.bz2 ; \
tar xjf ./jemalloc.tar.bz2 ; \
cd jemalloc-5.3.1 ; \
...
Two problems:
;instead of&&. Whencurl -fexits non-zero, the recipe continues intotar xjfon a file that does not exist.curl -salso suppresses the reason it failed, so nothing about the actual failure is ever printed.- No retry. The
Dockerfilealready treats this class of failure as expected and retries —apk add --no-cache ... || (sleep 5 && apk add --no-cache ...), commented "Retry once on transient apk-proxy errors". The jemalloc fetch gets no equivalent.
Observed failure
What a maintainer actually sees, with the genuine cause absent:
Downloading jemalloc
tar (child): ./jemalloc.tar.bz2: Cannot open: No such file or directory
tar (child): Error is not recoverable: exiting now
tar: Child returned status 2
/bin/sh: 6: cd: can't cd to jemalloc-5.3.1
sed: can't read src/jemalloc_cpp.cpp: No such file or directory
/bin/sh: 8: ./configure: not found
make[2]: *** No targets specified and no makefile found. Stop.
==== Need sudo access to install jemalloc
make: *** No rule to make target 'install'. Stop.
make[1]: *** [Makefile:111: jemalloc] Error 2
The tar: Cannot open line reads like a missing-file bug, which sends you looking in the wrong place. Every line after it is downstream noise.
We hit this four times across two PRs in a fork of this repo, in both the host-native build and the RUN make -C dgraph jemalloc layer of the Docker build. Because the target is a prerequisite of make dgraph, it takes out every CI job that builds a binary — for us that was unit, systest-baseline, and any suite depending on them.
Suggested fix
curl -fSL ${JEMALLOC_URL} -o jemalloc.tar.bz2 \
|| (sleep 5 && curl -fSL ${JEMALLOC_URL} -o jemalloc.tar.bz2) ; \
tar xjf ./jemalloc.tar.bz2 && \
cd jemalloc-5.3.1 && \
...
&&between the steps that depend on each other, so the first real failure is the one reported.-S(keeping-f) so curl prints why it failed; dropping-sentirely also works.- One retry with a short sleep, matching the convention the Dockerfile already uses.
Caching the tarball would remove the network dependency altogether, but the retry plus honest error reporting is the small fix.
I'm happy to open a PR for this if it's welcome.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in dgraph/Makefile at the jemalloc target and compare its download flow with the retry convention shown in Dockerfile. Run the jemalloc target or the Docker build to observe the current failure path. Done means download errors remain visible, one transient retry occurs, and extraction and later steps stop when a prerequisite fails.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, shell
- Domain
- build-system, devops
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 84/100