apache / apache/cloudberry

ic-proxy: interconnect error: connection closed prematurely

Open
#359 2 comments 0 reactions 0 assignees View on GitHub
type: Bug
Dominant language
C
Stars
1.4k
Forks
247
Avg merge
4d 3h
Merged PRs (30d)
39

Description

### Cloudberry Database version

postgres=# select version();
version

--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
-----------------------------------------------------------------
PostgreSQL 14.4 (Cloudberry Database 1.4.0 build commit:e83e3ffc22d538deb2dbceeeae0138ca2de064e6) on x86_64-pc-linux-gnu, compiled by gcc (GCC) 8.3.1 20191121 (Anolis 8.3.1-[5.0.1.1](http://5.0.1.1/))
, 64-bit compiled on Sep 17 2023 23:10:22 (with assert checking)
(1 row)

### What happened

interconnect type set as proxy and received below errors:

master:

```
2024-01-08 12:18:03.418100 CST,",p493382,th-1664801920,""46890",2024-01-08 10:17:46 CST,0,con159,cmd23,seg-1,,dx6366183,,sx1,"ERROR","58M01","interconnect error: connection closed prematurely (seg19 xxx.xxx.xxx.xxx:6003 pid=39387)","from Remote Connection: contentId=60 at",,,,,"insert into xxxx select starttime ,period

```

segment:

```
2024-01-08 13:01:37.711883 CST,,,p45134,th235180928,,,,0,,,seg19,,,,,"WARNING","01000","peer[seg60,dbid62 xxx.xxx.xxx.xxx:16003->xxx.xxx.xxx.xxx:55071]: fail to shutdown: socket is not connected",,,,,,,0,,"ic_proxy_peer.c",446,
2024-01-08 13:01:37.713647 CST,,,p45134,th235180928,,,,0,,,seg19,,,,,"WARNING","01000","client[con64863,cmd3,slice[1->0] seg19:dbid21:p67661->seg48:dbid50:p640943]: fail to receive c2p DATA: connection reset by peer",,,,,,,0,,"ic_proxy_client.c",502,
2024-01-08 13:01:37.713728 CST,,p67661,th235180928,","10265",2024-01-08 13:01:09 CST,1909017,con64863,cmd3,seg19,,dx6366775,x1909018,sx2,"ERROR","58M01","interconnect error: connection closed prematurely","from Remote Connection: contentId=60 at",,,,,"insert into xxxxxx ^M
select a.starttime,^M

```

gp_interconnect_proxy_addresses port : 16003

### What you think should happen instead

_No response_

### How to reproduce

it is reproducing easily.

### Operating System

BELinux.

### Anything else

_No response_

### Are you willing to submit PR?

- [ ] Yes, I am willing to submit a PR!

### Code of Conduct

- [X] I agree to follow this project's [Code of Conduct](https://cloudberrydb.org/community/coc).

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the proxy interconnect failure and correlate the master and segment logs. Inspect ic_proxy_peer.c around line 446 and ic_proxy_client.c around line 502, then trace the connection shutdown and reset path. Done means the reproducible query no longer reports a premature connection close and the relevant behavior is covered by a regression test.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, postgresql
Domain
databases, distributed-systems, networking
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.