DefangLabs / DefangLabs/defang
`defang tail` stops streaming events mid-deployment on GCP provider
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 166
- Forks
- 24
- Avg merge
- 10h 8m
- Merged PRs (30d)
- 33
Description
Description
defang tail stops delivering log events partway through a GCP deployment while the deployment continues to make progress. The stream appears to hang silently. No error, no disconnect, the process stays alive. But no new events are delivered.
Steps to Reproduce
- Deploy a multi-service stack (4 services: api, frontend, worker + managed db/redis) to GCP using
defang compose up --detach - Run
defang tail --stack=betato follow the deployment - Observe that log streaming stops mid-deployment
Expected Behavior
defang tail should continue streaming events for the entire deployment lifecycle, including:
- Cloud Build completion for all services
- Cloud Run service updates and revision readiness
- Pulumi resource creation (NEG, backend service, URL map, HTTPS proxy, forwarding rule)
- Worker instance group updates
- Final "Update succeeded/failed" message
Actual Behavior
defang tail streams initial events (Pulumi update start, Cloud Build progress, early apt-get output from Docker builds) but then stops around the 1-2 minute mark. The last events received were Cloud Build Docker layer pulls and apt-get package resolution for the API image build.
Meanwhile, the deployment continued successfully for another ~6+ minutes in GCP:
- All three Cloud Build jobs completed
- API service ran Django migrations and started gunicorn successfully
- Both Cloud Run services (api, frontend) reached
CONDITION_SUCCEEDED - Load balancer resources (NEG, backend service, URL map, HTTPS proxy, forwarding rule) were all created
- Worker instance group was updated
None of these events appeared in defang tail. The process remained alive with no error output — just silent.
I confirmed the deployment was progressing by querying the GCP APIs directly (Cloud Run services API, Compute Engine instance group manager API), which showed healthy services while defang tail showed nothing.
Environment
- Defang CLI: v3.3.2
- Defang Fabric: v0.6.0-1519-g598a4a1d
- Provider: GCP (
DEFANG_PROVIDER=gcp) - GCP Project:
ecgpt-a9bf8 - Region: us-central1
- Stack: beta
- Deployment ID:
ang5tnyb63sp - OS: Linux
Additional Context
defang compose up(non-detached, with inline log tailing) appeared to have the same issue in a prior deployment attempt: it showed early build logs but stopped delivering events before the deployment completed or failed.- The
defang tailprocess never exited or printed an error; it just stopped receiving events while still appearing connected. defang lsalso returned "No active deployments found" during the deployment, which may be a related issue with deployment state visibility.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the deployment with defang compose up --detach, then follow it with defang tail --stack=beta and compare the output with GCP's Cloud Run and Compute Engine APIs. Also check whether defang ls reports the deployment during this period. Done means defang tail remains connected and reports events through the final success or failure message.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker-compose, go, google-cloud
- Domain
- cli, cloud
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100