ic-proxy: interconnect error: connection closed prematurely
- Dominant language
- C
- Stars
- 1.4k
- Forks
- 247
- Avg merge
- 4d 3h
- Merged PRs (30d)
- 39
Description
### Cloudberry Database version
postgres=# select version();
version
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
-----------------------------------------------------------------
PostgreSQL 14.4 (Cloudberry Database 1.4.0 build commit:e83e3ffc22d538deb2dbceeeae0138ca2de064e6) on x86_64-pc-linux-gnu, compiled by gcc (GCC) 8.3.1 20191121 (Anolis 8.3.1-[5.0.1.1](http://5.0.1.1/))
, 64-bit compiled on Sep 17 2023 23:10:22 (with assert checking)
(1 row)
### What happened
interconnect type set as proxy and received below errors:
master:
```
2024-01-08 12:18:03.418100 CST,",p493382,th-1664801920,""46890",2024-01-08 10:17:46 CST,0,con159,cmd23,seg-1,,dx6366183,,sx1,"ERROR","58M01","interconnect error: connection closed prematurely (seg19 xxx.xxx.xxx.xxx:6003 pid=39387)","from Remote Connection: contentId=60 at",,,,,"insert into xxxx select starttime ,period
```
segment:
```
2024-01-08 13:01:37.711883 CST,,,p45134,th235180928,,,,0,,,seg19,,,,,"WARNING","01000","peer[seg60,dbid62 xxx.xxx.xxx.xxx:16003->xxx.xxx.xxx.xxx:55071]: fail to shutdown: socket is not connected",,,,,,,0,,"ic_proxy_peer.c",446,
2024-01-08 13:01:37.713647 CST,,,p45134,th235180928,,,,0,,,seg19,,,,,"WARNING","01000","client[con64863,cmd3,slice[1->0] seg19:dbid21:p67661->seg48:dbid50:p640943]: fail to receive c2p DATA: connection reset by peer",,,,,,,0,,"ic_proxy_client.c",502,
2024-01-08 13:01:37.713728 CST,,p67661,th235180928,","10265",2024-01-08 13:01:09 CST,1909017,con64863,cmd3,seg19,,dx6366775,x1909018,sx2,"ERROR","58M01","interconnect error: connection closed prematurely","from Remote Connection: contentId=60 at",,,,,"insert into xxxxxx ^M
select a.starttime,^M
```
gp_interconnect_proxy_addresses port : 16003
### What you think should happen instead
_No response_
### How to reproduce
it is reproducing easily.
### Operating System
BELinux.
### Anything else
_No response_
### Are you willing to submit PR?
- [ ] Yes, I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://cloudberrydb.org/community/coc).
Contributor guide
Research direction
Start by reproducing the proxy interconnect failure and correlate the master and segment logs. Inspect ic_proxy_peer.c around line 446 and ic_proxy_client.c around line 502, then trace the connection shutdown and reset path. Done means the reproducible query no longer reports a premature connection close and the relevant behavior is covered by a regression test.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- c, postgresql
- Domain
- databases, distributed-systems, networking
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100