livecycle / livecycle/preevy

[Bug]: Multiple environments with same ID causes opaque tunnel error

Open
#456 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
TypeScript
Stars
2.2k
Forks
91
PR merge metrics
No merged PRs in 30d

Description

What happened?

Somehow I ended up with multiple preevy environments with the same ID. This caused the tunnel to fail, but it wasn't clear why.

Add screenshots

In the GitHub actions log, I saw this:

- Getting tunnel URLs...
✔ Getting tunnel URLs...
Writing URLs to file /home/runner/work/_temp/preevy_urls.8487.json
 Service Port URL 
 ─────── ──── ─── 
Raw URLs JSON
[]
URLs map
null

A comment with no URLs was added to the PR:

SCR-20240325-ioqo

Looking at docker, I saw the proxy container was crashing. This was in the container log:

[13:26:34.110] WARN (1): shutting down on uncaughtException: Error: Unable to bind to /preevy_proxy-maybern-preview-eric-preevy#access=private;meta=xxxxxxxxxxxxx
    at file:///app/index.mjs:18156:49
    at REQUEST_FAILURE (file:///app/index.mjs:17365:34)
    at 82 (file:///app/index.mjs:9616:21)
    at Protocol.onPayload (file:///app/index.mjs:13200:14)
    at AESGCMDecipherNative.decrypt (file:///app/index.mjs:6321:30)
    at Protocol.parsePacket [as _parse] (file:///app/index.mjs:13178:29)
    at Protocol.parse (file:///app/index.mjs:11948:20)
    at TLSSocket.<anonymous> (file:///app/index.mjs:17584:25)
    at TLSSocket.emit (node:events:517:28)
    at addChunk (node:internal/streams/readable:368:12), 'unhandledRejection'
Steps to reproduce the behavior

I am not sure how I ended up in this state.

Expected behavior

Error message that explains what went wrong and how to fix it.

In my case I fixed the issue by using kubectl get statefulsets get a list and then kubectl delete statefulset xxx to get rid of the duplicates. Thanks @royra on Slack for your help!

What OS are you seeing the problem on?

Linux

Additional context

No response

Record
  • I agree to follow this project's Code of Conduct

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

The payload names no source file or test; start by tracing the tunnel/proxy failure represented by /app/index.mjs and inspect the duplicate StatefulSets with kubectl. Compare the empty tunnel URL output with the proxy bind error. Done means the user-facing error identifies duplicate environment IDs and explains how to remove the duplicates.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, kubernetes, typescript
Domain
cli, devops, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.