cloudfoundry / cloudfoundry/cloud_controller_ng

Error staging application while updating api vm during update

Open
#2,839 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

unscheduled
Dominant language
Ruby
Stars
207
Forks
373
Avg merge
2d 12h
Merged PRs (30d)
56

Description

Issue

We are testing the cf-update with a job, that is doing the update with a cf-push every minute during the update process so we can be sure there a no regressions introduced.
During this process we get occasionally one specific error.

Context

During the update of the api vms, we occasionally get the error:

App lifecycle operation error: cmd=['cf8', 'push', 'cf_downtime_push_app', '-b', 'staticfile_buildpack', '-m', '64M', '--no-manifest'], error=Error staging application cf_downtime_push_app: timed out after 1 minute(s)

This error occurs, when a cf-push operation is running while api jobs are getting drained:
executing drain: api/GUID

Looking into the drain logs, we could not find any abnormalities. Also stderr and stdout logs during that time did not show an error.

Steps to Reproduce

Run an update where api vms need to get updated with simultaneously running cf-push operations.

Expected result

We would expect that there are no errors during the update of the vm and that the jobs are getting drained properly

Current result

Occasional errors while draining api vm

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing a cf-push while API VMs are updating and draining, then compare the staging, drain, stderr, and stdout logs around the timeout. Done means the concurrent update completes without staging errors and API jobs drain properly.

Written by the indexing model from the issue text.

Assessment

Tech stack
ruby
Domain
api, backend, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.