gpaddmirrors & gprecoverseg issue

Hi,

I have started adding mirrors as 16 primary segments. After completing one mirror segment (100%) pg_basebackup removed the mirror segment folder started again from scratch and same behavior for the rest of the segments.

Then I used gprecoverseg with -F option against only 4 segments and again observed the same behavior.

PFB the snapshot:

sky-cbseg03 (dbid 19): 1034761642/1165354896 kB (88%), 0/1 tablespace (...rrors/gpseg0/base/17018/280446.1)
sky-cbseg03 (dbid 20): pg_basebackup: removing data directory "/data/cbdatabase/mirrors/gpseg1"
sky-cbseg03 (dbid 21): pg_basebackup: removing data directory "/data/cbdatabase/mirrors/gpseg2"
sky-cbseg03 (dbid 22): 1026698590/1162146943 kB (88%), 0/1 tablespace (...mirrors/gpseg3/base/17018/280677)

20260330:16:18:53:1407854 gpstate:sky-cbcoord:gpadmin-[INFO]:-local Cloudberry Version: 'postgres (Apache Cloudberry) 2.0.0-incubating build 1'
20260330:16:18:53:1407854 gpstate:sky-cbcoord:gpadmin-[INFO]:-coordinator Cloudberry Version: 'PostgreSQL 14.4 (Apache Cloudberry 2.0.0-incubating build 1) on x86_64-pc-linux-gnu, compiled by gcc (GCC) 11.5.0 20240719 (Red Hat 11.5.0-5.0.1), 64-bit compiled on Aug 28 2025 15:25:48'
20260330:16:18:53:1407854 gpstate:sky-cbcoord:gpadmin-[INFO]:-Obtaining Segment details from coordinator...
20260330:16:18:53:1407854 gpstate:sky-cbcoord:gpadmin-[INFO]:-Gathering data from segments...
..
20260330:16:18:55:1407854 gpstate:sky-cbcoord:gpadmin-[INFO]:-Cloudberry instance status summary
20260330:16:18:55:1407854 gpstate:sky-cbcoord:gpadmin-[INFO]:-----------------------------------------------------
20260330:16:18:55:1407854 gpstate:sky-cbcoord:gpadmin-[INFO]:- Coordinator instance = Active
20260330:16:18:55:1407854 gpstate:sky-cbcoord:gpadmin-[INFO]:- Coordinator standby = No coordinator standby configured
20260330:16:18:55:1407854 gpstate:sky-cbcoord:gpadmin-[INFO]:- Total segment instance count from metadata = 32
20260330:16:18:55:1407854 gpstate:sky-cbcoord:gpadmin-[INFO]:-----------------------------------------------------
20260330:16:18:55:1407854 gpstate:sky-cbcoord:gpadmin-[INFO]:- Primary Segment Status
20260330:16:18:55:1407854 gpstate:sky-cbcoord:gpadmin-[INFO]:-----------------------------------------------------
20260330:16:18:55:1407854 gpstate:sky-cbcoord:gpadmin-[INFO]:- Total primary segments = 16
20260330:16:18:55:1407854 gpstate:sky-cbcoord:gpadmin-[INFO]:- Total primary segment valid (at coordinator) = 16
20260330:16:18:55:1407854 gpstate:sky-cbcoord:gpadmin-[INFO]:- Total primary segment failures (at coordinator) = 0
20260330:16:18:55:1407854 gpstate:sky-cbcoord:gpadmin-[INFO]:- Total number of postmaster.pid files missing = 0
20260330:16:18:55:1407854 gpstate:sky-cbcoord:gpadmin-[INFO]:- Total number of postmaster.pid files found = 16
20260330:16:18:55:1407854 gpstate:sky-cbcoord:gpadmin-[INFO]:- Total number of postmaster.pid PIDs missing = 0
20260330:16:18:55:1407854 gpstate:sky-cbcoord:gpadmin-[INFO]:- Total number of postmaster.pid PIDs found = 16
20260330:16:18:55:1407854 gpstate:sky-cbcoord:gpadmin-[INFO]:- Total number of /tmp lock files missing = 0
20260330:16:18:55:1407854 gpstate:sky-cbcoord:gpadmin-[INFO]:- Total number of /tmp lock files found = 16
20260330:16:18:55:1407854 gpstate:sky-cbcoord:gpadmin-[INFO]:- Total number postmaster processes missing = 0
20260330:16:18:55:1407854 gpstate:sky-cbcoord:gpadmin-[INFO]:- Total number postmaster processes found = 16
20260330:16:18:55:1407854 gpstate:sky-cbcoord:gpadmin-[INFO]:-----------------------------------------------------
20260330:16:18:55:1407854 gpstate:sky-cbcoord:gpadmin-[INFO]:- Mirror Segment Status
20260330:16:18:55:1407854 gpstate:sky-cbcoord:gpadmin-[INFO]:-----------------------------------------------------
20260330:16:18:55:1407854 gpstate:sky-cbcoord:gpadmin-[INFO]:- Total mirror segments = 16
20260330:16:18:55:1407854 gpstate:sky-cbcoord:gpadmin-[INFO]:- Total mirror segment valid (at coordinator) = 0
20260330:16:18:55:1407854 gpstate:sky-cbcoord:gpadmin-[WARNING]:-Total mirror segment failures (at coordinator) = 16 <<<<<<<<
20260330:16:18:55:1407854 gpstate:sky-cbcoord:gpadmin-[WARNING]:-Total number of postmaster.pid files missing = 16 <<<<<<<<
20260330:16:18:55:1407854 gpstate:sky-cbcoord:gpadmin-[INFO]:- Total number of postmaster.pid files found = 0
20260330:16:18:55:1407854 gpstate:sky-cbcoord:gpadmin-[WARNING]:-Total number of postmaster.pid PIDs missing = 16 <<<<<<<<
20260330:16:18:55:1407854 gpstate:sky-cbcoord:gpadmin-[INFO]:- Total number of postmaster.pid PIDs found = 0
20260330:16:18:55:1407854 gpstate:sky-cbcoord:gpadmin-[WARNING]:-Total number of /tmp lock files missing = 16 <<<<<<<<
20260330:16:18:55:1407854 gpstate:sky-cbcoord:gpadmin-[INFO]:- Total number of /tmp lock files found = 0
20260330:16:18:55:1407854 gpstate:sky-cbcoord:gpadmin-[WARNING]:-Total number postmaster processes missing = 16 <<<<<<<<
20260330:16:18:55:1407854 gpstate:sky-cbcoord:gpadmin-[INFO]:- Total number postmaster processes found = 0
20260330:16:18:55:1407854 gpstate:sky-cbcoord:gpadmin-[INFO]:- Total number mirror segments acting as primary segments = 0
20260330:16:18:55:1407854 gpstate:sky-cbcoord:gpadmin-[INFO]:- Total number mirror segments acting as mirror segments = 16
20260330:16:18:55:1407854 gpstate:sky-cbcoord:gpadmin-[INFO]:-----------------------------------------------------

pg_basebackup: error: connection to server at "sky-cbseg04" (10.247.224.64), port 50002 failed: Connection refused
Is the server running on that host and accepting TCP/IP connections?

Can you guide why it is behaving like this ?

Start with the gpaddmirrors and gprecoverseg -F workflows, then trace the pg_basebackup output for the mirror data directories and the connection-refused error on sky-cbseg04. Compare the recovery logs with the gpstate mirror status; done means identifying why completed mirror directories are removed and documenting or correcting the recovery behavior.

Written by the indexing model from the issue text.