deis / deis/builder

Builder proceeds if slugrunner pod is evicted

Open
#496 3 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Go
Stars
40
Forks
41
PR merge metrics
No merged PRs in 30d

Description

My slugrunner pod is quite often evicted due to low compute resources on the node.

An example log of a git push of a buildpack build.

```
[chexxor@fedora myapp]$ git push ssh://git@deis-builder.123.456.789.012.nip.io:2222/myapp-master.git master
Counting objects: 4129, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (3757/3757), done.
Writing objects: 100% (4129/4129), 5.39 MiB | 4.75 MiB/s, done.
Total 4129 (delta 2821), reused 474 (delta 288)
remote: Resolving deltas: 100% (2821/2821), done.
Starting build... but first, coffee!
-----> Restoring cache...
Done!
-----> Node.js app detected

-----> Creating runtime environment

NPM_CONFIG_LOGLEVEL=error
NPM_CONFIG_PRODUCTION=true
NODE_ENV=production
NODE_MODULES_CACHE=false

-----> Installing binaries
engines.node (package.json): 4.6.0
engines.npm (package.json): 2.15.x

Downloading and installing node 4.6.0...
Resolving npm version 2.15.x via semver.io...
Downloading and installing npm 2.15.11 (replacing version 2.15.9)...

-----> Restoring cache
Skipping cache restore (disabled by config)

-----> Building dependencies
Running heroku-prebuild

> rentable@1.0.0 heroku-prebuild /tmp/build
> echo "Prebuild steps running..."

Installing node modules (package.json)
Build complete.
Launching App...
...
...
...
...
...ote:
...
...
...
...ote:
...
...
...
...
...
...
...
...
...
...
...
...
Done, myapp-master:v76 deployed to Workflow
```

Note that the slugbuilder pod was evicted while executing the `-----> Building dependencies` step. I believe this because the logs produced by this step *should be* hundreds of lines, and the following buildpack steps don't appear, like ["-----> Caching build"](https://github.com/heroku/heroku-buildpack-nodejs/blob/master/bin/compile#L169) and ["-----> Build succeeded!"](https://github.com/heroku/heroku-buildpack-nodejs/blob/master/bin/compile#L178).

Despite this slugbuilder pod failing, the builder process continues, and prints ["Build complete.](https://github.com/deis/builder/blob/4634fe14784c954027af167d15ff8d3490cf4942/pkg/gitreceive/build.go#L306) skipping the [failed pod check](https://github.com/deis/builder/blob/4634fe14784c954027af167d15ff8d3490cf4942/pkg/gitreceive/build.go#L293-L304).

I upgraded my workflow just a few days ago, so I believe I have the latest versions of these components.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.