Shopify / Shopify/shipit-engine

Continuous Deployment silently failing with 0.23.1

Open
#805 7 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Ruby
Stars
1.5k
Forks
154
Avg merge
2h 26m
Merged PRs (30d)
3

Description

One of our stacks is configured for continuous deployment which worked fine under 0.20 and 0.21, but after upgrading to 0.23 (and then 0.23.1), continuous deployment seems to have silently stopped working.

I can find no errors in the production.log or sidekiq.log, and jobs seem to be enqueuing normally. Manual deployments all continue to work fine.

If I look at the shipit.stacks table in our database, things look normal - pulling out some things that look related:

lock_reason: NULL
tasks_count: 1241
continuous_deployment: 1
undeployed_commits_count: 1
lock_author_id: NULL
ignore_ci: 0
continuous_delivery_delayed_since: NULL
locked_since: NULL
merge_queue_enabled: 0
last_deployed_at: 2018-06-20 01:47:52

With nothing obviously erroring I'm at a bit of a loss as to figure out why it's stopped working. Any suggestions on what I could check?

Thanks!

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No source file or test is named. Start by tracing continuous-deployment handling from the enqueued jobs and compare it with the working manual-deployment path, using production.log, sidekiq.log, and the shipit.stacks fields as clues. Done means identifying why deployment stops silently after 0.23.1 and adding a verified fix or diagnostic.

Written by the indexing model from the issue text.

Assessment

Tech stack
ruby
Domain
devops, release
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.