docker-library / docker-library/postgres

Document / adjust defaults for possible `connection (already) closed` issue when used on Swarm (due to IPVS)

Open
#1,110 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Shell
Stars
2.5k
Forks
1.2k
PR merge metrics
No merged PRs in 30d

Description

The issue https://github.com/docker-library/postgres/issues/538 introduced a warning in the README about the Docker Swarm IPVS LB that timeouts TCP connections after 900 secs, which is lower than tcp_keepalive_time, so idle connections might become unavailable (cut by IPVS) while still being needed later, and that warning have since been removed (https://github.com/docker-library/docs/commit/5e28015ab2d9039a28daca5f7d65be996eb39234), probably due to the link being broken.

Could it be possible to reintroduce a warning about this, and/or to propose a default -ctcp_keepalives_idle=870 value (for example; and maybe tcp_keepalives_interval+tcp_keepalives_count)?

The old "success" documentation is visible at https://web.archive.org/web/20200611114911/https://success.docker.com/article/ipvs-connection-timeout-issue.

Possible solutions are:

  • connect to postgres without the LB using tasks.<service_name>
    • client must resolve on each connection in case the IP changed
  • use endpoint_mode: dnsrr to prevent using the LB
    • client must resolve on each connection in case the IP changed
  • use net.ipv4.tcp_keepalive_time: 870 (<900)
  • use -ctcp_keepalives_idle=870 (<900)

Example of all solutions (only one needed):

services:
  postgres:
    command:
      - postgres
      - -ctcp_keepalives_idle=300  # < 900
      # Maybe this too ?
      # - -ctcp_keepalives_interval=30
      # - -ctcp_keepalives_count=5
    sysctls:
      net.ipv4.tcp_keepalive_time: 720  # < 900
    deploy:
      endpoint_mode: dnsrr  # The client should resolve on each connection in case the task (IP) changed

  my_other_service:
    environment:
      POSTGRES_HOST: tasks.postgres  # The client should resolve on each connection in case the task (IP) changed

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the Postgres README and review issue 538, the removed documentation commit, and the archived Docker Success article. Compare the proposed Swarm/IPVS connection strategies and determine what warning or default guidance is appropriate. Done means the README clearly documents the risk and supported mitigations, including any agreed keepalive defaults.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, postgresql
Domain
databases, documentation, networking
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.