raystack / raystack/frontier

Share one postgres container across the internal/store/postgres test suites

Open
#1,932 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement go
Dominant language
Go
Stars
344
Forks
47
Avg merge
4d 4h
Merged PRs (30d)
26

Description

What happens now

Every test suite in internal/store/postgres starts its own postgres container. 24 test files call newTestClient from their SetupSuite, and each call pulls up a postgres:13 container, waits for it to accept connections, drops and recreates the public schema, and runs the full migration set. TearDownSuite then throws the container away.

make test runs with -count 2, so that is 48 container starts and 48 migration runs for one CI job.

There is no TestMain in the package today, so there is nowhere for a shared fixture to live.

Why it is worth fixing

The package is slow and it sits right on the edge of its own timeout.

Measured locally with the CI flags (-race -count 2), the package takes about 120s. Container startup is most of that. On CI it is slower, and it recently started failing outright:

panic: test timed out after 2m30s
FAIL	github.com/raystack/frontier/internal/store/postgres	150.221s

The budget was -timeout 150s and the package used 150.2s of it. Adding a single new suite was enough to tip it over, because the remaining headroom was about 3s. The same timeout panic showed up on several pushes before that new suite existed, so it had been flaky at the limit for a while.

PR #1929 raised -timeout to 600s in the Makefile to unblock CI. That is a workaround. It buys room but does not make the package faster, and the next few suites will eat the new headroom the same way.

Suggested direction

Start one postgres container for the whole package and give each suite its own isolated database or schema on it.

  • Add a TestMain in postgres_test that starts the container once, runs the migrations once, and tears it down at the end.
  • Give each suite its own database (CREATE DATABASE) or its own schema, which costs milliseconds instead of seconds.
  • Keep newTestClient's signature, or replace it with a helper that hands back a client pointed at a fresh database, so the 24 suites need only a small mechanical change each.

That should turn 48 container starts into 1 and cut minutes off every run of the unit job.

One related trap

newTestClient calls resource.Expire(120), which tells docker to hard kill the container after 120 seconds. That was harmless while the whole package had to finish in 150s. Now that the timeout is 600s, any suite that runs longer than 120 seconds will have its database killed underneath it, and the failure will look like a connection error rather than a timeout. Worth handling in the same change.

Pointers

  • internal/store/postgres/postgres_test.go - newTestClient, setup, purgeDocker
  • Makefile - the test target and its -count 2 -timeout 600s
  • .github/workflows/test.yml - the unit job that runs make test

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with internal/store/postgres/postgres_test.go, especially newTestClient, setup, and purgeDocker, then inspect the 24 suites' SetupSuite and TearDownSuite methods. Read the Makefile test target and .github/workflows/test.yml to understand the race, count, and timeout settings. Done means the package uses one container and one migration run while each suite remains isolated and make test passes without the 120-second expiry trap.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, go, postgresql
Domain
databases, devops, testing
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
62/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.