PostgreSQL not starting correctly if image already contains data

Hi everyone. Reporting this after my TestContainers talk at Jfokus and talking about this with @kiview.

Description

If the image you are running with already contains data in the database (because it was built for that specific reason... with built in testdata or with fully ran migrations), then the TestContainer doesn't recognize correctly that the database has started.

The reason for this is the default WaitStrategy, which waits until "database system is ready" is logged 2 times.

// From PostgreSQLContainer.class
this.waitStrategy = (new LogMessageWaitStrategy())
                                         .withRegEx(".*database system is ready to accept connections.*\\s")
                                         .withTimes(2)
                                         .withStartupTimeout(Duration.of(60L, ChronoUnit.SECONDS));

This waiter is correct when running with an empty database. Starting a new container like below results in a logfile with 2x that log line.

@Container
public static PostgreSQLContainer postgreSQLContainer = new PostgreSQLContainer<>("postgres:14.2") // database image without data.
       .withDatabaseName("testcontainer")
       .withUsername("sa")
       .withPassword("sa");

Results in the following logs: postgresdb-no-data.txt

However, when you build a custom image which includes some data (like I have done below), then the logs will only contain "database system is ready" a single time.

private static DockerImageName IMAGE = DockerImageName.parse("tomcools/postgres:dev")
            .asCompatibleSubstituteFor("postgres");

@Container
 public static PostgreSQLContainer postgreSQLContainer = new PostgreSQLContainer<>(IMAGE)
       .withDatabaseName("testcontainer")
       .withUsername("sa")
       .withPassword("sa");

postgresdb-with-data-test-logs.txt

Workaround

The way I have worked around this for now, it to change the default waiter with a custom one that only waits until the log has passed a single time.

    private static DockerImageName IMAGE = DockerImageName.parse("tomcools/postgres:dev")
            .asCompatibleSubstituteFor("postgres");

    @Container
    public static PostgreSQLContainer postgreSQLContainer = new PostgreSQLContainer<>(IMAGE)
            // Custom waiter
            .waitingFor((new LogMessageWaitStrategy())
                    .withRegEx(".*database system is ready to accept connections.*\\s")
                    .withTimes(1)
                    .withStartupTimeout(Duration.of(60L, ChronoUnit.SECONDS))
            )

Possible solution directions

The contribution documentation states that, in order for something to become a module, it needs to "add value", where one of the examples given is:

does it add technology-specific wait strategies?

Given that statement, I'd expect the PostgreSQLContainer to be bootstrapped with a waiter that can handle both images with or without data present.

Possible ideas:

• Create a method to set "withData(boolean)" , which can setup a different waiter, but that doesn't feel intuitive;
• Allow composing of WaitStrategy(s): might need some AND/OR/NOT logic then, this would allow the creation of more complex wait strategy here to either check for 2x original log line (empty db), or 1x "data present" + 1x original log line;
• Create a JDBC polling wait strategy, where you try to establish a JDBC connection (we already have the JDBC connection url), or potentially even execute a test query;
• Something else???

I'm willing to help implement this. If no solution is implemented, I'd say we at least document the workaround on the PostgreSQL page.

Kind regards,
Tom

Start in PostgreSQLContainer.class and read docs/features/startup_and_waits.md, then compare the supplied logs for images with and without pre-existing data. Define and test a startup check that handles both log patterns, or document the workaround if that is the chosen scope.