salesforce / salesforce/rules_docker_compose_test

docker_compose_test returns 0 (success) even when dependent service fails to start

Open
#41 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Starlark
Stars
7
Forks
5
Avg merge
18h 49m
Merged PRs (30d)
3

Description

If we have service_a which depends on service_b being healthy, and service_b exits right away with a non-zero exit code, docker compose will bring down the services and report success (exit code 0). The reason being that service_a never reaches the "Started" state. So, --exit-code-from returns zero.

Here is a possible (untested) diff that might help fix the issue:

--- docker_compose_test.sh.orig
+++ docker_compose_test.sh
@@ -51,9 +51,21 @@
 echo "$docker_compose_up_cmd" | bash
 result=$?
 
-# Figure out the exit code of the test container incase it never actually started.
-EXIT_CODE=$(docker inspect $("${docker_compose_bin[@]}" -f $ABSOLUTE_COMPOSE_FILE_PATH ps -qa $DOCKER_COMPOSE_TEST_CONTAINER) --format='{{.State.ExitCode}}' 2>/dev/null)
-if [ "$EXIT_CODE" != "0" ] || [ -z "$EXIT_CODE" ]; then
-  echo "Error: $DOCKER_COMPOSE_TEST_CONTAINER container failed or never started!"
+# `docker compose up --exit-code-from` can still exit 0 in edge cases (e.g. the named service never
+# schedules while Compose treats the session as done). Resolve the service container via this compose
+# project and verify it actually exited successfully.
+SERVICE="$DOCKER_COMPOSE_TEST_CONTAINER"
+CID=$("${docker_compose_bin[@]}" -f "$ABSOLUTE_COMPOSE_FILE_PATH" ps -a -q "$SERVICE" 2>/dev/null | head -n 1) || CID=""
+
+EXIT_CODE="$( [ -n "$CID" ] && docker inspect "$CID" --format '{{.State.ExitCode}}' 2>/dev/null || echo "" )"
+STATUS="$( [ -n "$CID" ] && docker inspect "$CID" --format '{{.State.Status}}' 2>/dev/null || echo "not-found" )"
+
+# We must verify the container reached the 'exited' state. 
+# Containers in 'created' or 'running' state often have ExitCode 0, leading to false positives.
+if [ "$STATUS" = "exited" ] && [ "$EXIT_CODE" = "0" ]; then
+  echo "PASS ($SERVICE container exited with 0)"
+  exit 0
+fi
+
+echo "FAIL ($SERVICE status=$STATUS exit_code=${EXIT_CODE:-none} cid=${CID:-none})" >&2
 exit 1
-fi
-exit $result

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in docker_compose_test.sh around the docker compose up command and the existing container inspection. Reproduce the dependent-service failure described in the issue, then verify that a test service which never reaches an exited state is reported as a failure rather than success, while a successfully exited service with code 0 still passes.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, docker-compose, shell
Domain
devops, testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
64/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.