Unable to set up replication
- Dominant language
- Go
- Stars
- 2.1k
- Forks
- 73
- Avg merge
- 1d 10h
- Merged PRs (30d)
- 129
Description
I've tried with 2 types of OS without success, each with its own problem. Both are doltgres 0.7.5
Ubuntu 20.04:
config.yaml:
```yml
listener:
host: 127.0.0.1
port: 5455
postgres_replication:
postgres_server_address: 127.0.0.1
postgres_user: postgres
postgres_password: 123456
postgres_database: postgres
postgres_port: 5433
slot_name: doltgres_slot_test
behavior:
dolt_transaction_commit: true
log_level: debug
```
log of running `doltgres --data-dir /vol2/dolgresql/data`
```log
Starting replication
INFO[0000] Server ready. Accepting connections.
WARN[0000] secure_file_priv is set to "", which is insecure.
WARN[0000] Any user with GRANT FILE privileges will be able to read any file which the sql-server process can read.
WARN[0000] Please consider restarting the server with secure_file_priv set to a safe (or non-existent) directory.
INFO[0000] NewConnection DisableClientMultiStatements=false connectionID=1
INFO[0000] NewConnection DisableClientMultiStatements=false connectionID=2
INFO[0000] ConnectionClosed connectionID=1
DEBU[0000] Starting query connectTime="2024-05-31 08:05:51.039836801 +0000 UTC m=+0.077038967" connectionID=2 query="USE `doltgres`;"
DEBU[0000] Query finished in 0 ms connectTime="2024-05-31 08:05:51.039836801 +0000 UTC m=+0.077038967" connectionDb=doltgres connectionID=2 query="USE `doltgres`;"
2024/05/31 08:05:51 Starting replicator
2024/05/31 08:05:51 Starting logical replication on slot doltgres_slot_test at WAL location 0/1
2024/05/31 08:05:51 Logical replication started on slot doltgres_slot_test
2024/05/31 08:05:51 Primary Keepalive Message => ServerWALEnd: 0/1 ServerTime: 2024-05-31 08:05:51.047129 +0000 UTC ReplyRequested: false
2024/05/31 08:05:51 Primary Keepalive Message => ServerWALEnd: 4/C62022B8 ServerTime: 2024-05-31 08:05:51.047342 +0000 UTC ReplyRequested: false
2024/05/31 08:06:01 Error: write failed: write tcp 127.0.0.1:43912->127.0.0.1:5433: i/o timeout. Retrying
2024/05/31 08:06:01 Starting logical replication on slot doltgres_slot_test at WAL location 0/1
2024/05/31 08:06:01 Logical replication started on slot doltgres_slot_test
2024/05/31 08:06:01 Sent Standby status message with WALWritePosition = 0/1, WALApplyPosition = 4/C62022B9
2024/05/31 08:06:01 Primary Keepalive Message => ServerWALEnd: 0/1 ServerTime: 2024-05-31 08:06:01.047826 +0000 UTC ReplyRequested: false
2024/05/31 08:06:01 Primary Keepalive Message => ServerWALEnd: 4/C62022B8 ServerTime: 2024-05-31 08:06:01.049908 +0000 UTC ReplyRequested: false
2024/05/31 08:06:11 Error: write failed: write tcp 127.0.0.1:59418->127.0.0.1:5433: i/o timeout. Retrying
2024/05/31 08:06:11 Starting logical replication on slot doltgres_slot_test at WAL location 0/1
2024/05/31 08:06:11 Logical replication started on slot doltgres_slot_test
2024/05/31 08:06:11 Sent Standby status message with WALWritePosition = 0/1, WALApplyPosition = 4/C62022B9
2024/05/31 08:06:11 Primary Keepalive Message => ServerWALEnd: 4/C62022B8 ServerTime: 2024-05-31 08:06:11.057484 +0000 UTC ReplyRequested: false
2024/05/31 08:06:21 Sent Standby status message with WALWritePosition = 0/1, WALApplyPosition = 4/C62022B9
2024/05/31 08:06:21 Error: conn busy. Retrying
2024/05/31 08:06:21 Starting logical replication on slot doltgres_slot_test at WAL location 0/1
2024/05/31 08:06:21 Logical replication started on slot doltgres_slot_test
2024/05/31 08:06:21 Primary Keepalive Message => ServerWALEnd: 0/1 ServerTime: 2024-05-31 08:06:21.066913 +0000 UTC ReplyRequested: false
2024/05/31 08:06:21 Primary Keepalive Message => ServerWALEnd: 4/C62022B8 ServerTime: 2024-05-31 08:06:21.067196 +0000 UTC ReplyRequested: false
2024/05/31 08:06:31 Error: write failed: write tcp 127.0.0.1:39032->127.0.0.1:5433: i/o timeout. Retrying
2024/05/31 08:06:31 Starting logical replication on slot doltgres_slot_test at WAL location 0/1
2024/05/31 08:06:31 Logical replication started on slot doltgres_slot_test
2024/05/31 08:06:31 Sent Standby status message with WALWritePosition = 0/1, WALApplyPosition = 4/C62022B9
2024/05/31 08:06:31 Primary Keepalive Message => ServerWALEnd: 0/1 ServerTime: 2024-05-31 08:06:31.063825 +0000 UTC ReplyRequested: false
2024/05/31 08:06:31 Primary Keepalive Message => ServerWALEnd: 4/C62022B8 ServerTime: 2024-05-31 08:06:31.065772 +0000 UTC ReplyRequested: false
2024/05/31 08:06:41 Sent Standby status message with WALWritePosition = 0/1, WALApplyPosition = 4/C62022B9
2024/05/31 08:06:41 Error: conn busy. Retrying
2024/05/31 08:06:41 Starting logical replication on slot doltgres_slot_test at WAL location 0/1
2024/05/31 08:06:41 Logical replication started on slot doltgres_slot_test
2024/05/31 08:06:41 Primary Keepalive Message => ServerWALEnd: 0/1 ServerTime: 2024-05-31 08:06:41.072082 +0000 UTC ReplyRequested: false
2024/05/31 08:06:41 Primary Keepalive Message => ServerWALEnd: 4/C62022B8 ServerTime: 2024-05-31 08:06:41.072171 +0000 UTC ReplyRequested: false
2024/05/31 08:06:51 Sent Standby status message with WALWritePosition = 0/1, WALApplyPosition = 4/C62022B9
2024/05/31 08:06:51 Primary Keepalive Message => ServerWALEnd: 4/C62022B8 ServerTime: 2024-05-31 08:06:51.065432 +0000 UTC ReplyRequested: false
2024/05/31 08:07:01 Error: write failed: write tcp 127.0.0.1:56734->127.0.0.1:5433: i/o timeout. Retrying
2024/05/31 08:07:01 Starting logical replication on slot doltgres_slot_test at WAL location 0/1
2024/05/31 08:07:01 Logical replication started on slot doltgres_slot_test
2024/05/31 08:07:01 Sent Standby status message with WALWritePosition = 0/1, WALApplyPosition = 4/C62022B9
2024/05/31 08:07:01 Primary Keepalive Message => ServerWALEnd: 4/C62022B8 ServerTime: 2024-05-31 08:07:01.074074 +0000 UTC ReplyRequested: false
2024/05/31 08:07:11 Sent Standby status message with WALWritePosition = 0/1, WALApplyPosition = 4/C62022B9
2024/05/31 08:07:11 Primary Keepalive Message => ServerWALEnd: 4/C62022B8 ServerTime: 2024-05-31 08:07:11.074933 +0000 UTC ReplyRequested: false
2024/05/31 08:07:21 Sent Standby status message with WALWritePosition = 0/1, WALApplyPosition = 4/C62022B9
2024/05/31 08:07:21 Error: conn busy. Retrying
2024/05/31 08:07:21 Starting logical replication on slot doltgres_slot_test at WAL location 0/1
2024/05/31 08:07:21 Logical replication started on slot doltgres_slot_test
2024/05/31 08:07:21 Primary Keepalive Message => ServerWALEnd: 0/1 ServerTime: 2024-05-31 08:07:21.08955 +0000 UTC ReplyRequested: false
2024/05/31 08:07:21 Primary Keepalive Message => ServerWALEnd: 4/C62022B8 ServerTime: 2024-05-31 08:07:21.089704 +0000 UTC ReplyRequested: false
2024/05/31 08:07:31 Error: write failed: write tcp 127.0.0.1:51928->127.0.0.1:5433: i/o timeout. Retrying
2024/05/31 08:07:31 Starting logical replication on slot doltgres_slot_test at WAL location 0/1
2024/05/31 08:07:31 Logical replication started on slot doltgres_slot_test
2024/05/31 08:07:31 Sent Standby status message with WALWritePosition = 0/1, WALApplyPosition = 4/C62022B9
2024/05/31 08:07:31 Primary Keepalive Message => ServerWALEnd: 0/1 ServerTime: 2024-05-31 08:07:31.082044 +0000 UTC ReplyRequested: false
2024/05/31 08:07:31 Primary Keepalive Message => ServerWALEnd: 4/C62022B8 ServerTime: 2024-05-31 08:07:31.082326 +0000 UTC ReplyRequested: false
2024/05/31 08:07:41 Error: write failed: write tcp 127.0.0.1:39222->127.0.0.1:5433: i/o timeout. Retrying
2024/05/31 08:07:41 Starting logical replication on slot doltgres_slot_test at WAL location 0/1
2024/05/31 08:07:41 Logical replication started on slot doltgres_slot_test
2024/05/31 08:07:41 Sent Standby status message with WALWritePosition = 0/1, WALApplyPosition = 4/C62022B9
2024/05/31 08:07:41 Primary Keepalive Message => ServerWALEnd: 0/1 ServerTime: 2024-05-31 08:07:41.088457 +0000 UTC ReplyRequested: false
2024/05/31 08:07:41 Primary Keepalive Message => ServerWALEnd: 4/C62022B8 ServerTime: 2024-05-31 08:07:41.088672 +0000 UTC ReplyRequested: false
2024/05/31 08:07:51 Error during replication: write failed: write tcp 127.0.0.1:58328->127.0.0.1:5433: i/o timeout
2024/05/31 08:07:51 shutting down replicator
DEBU[0120] Received message: Terminate
INFO[0120] ConnectionClosed connectionID=2
```
I followed replication instruction here: https://docs.doltgres.com/guides/replication-from-postgres. Trying to test with `employees` table
=================================
MacOS M3 Max version 14.5
config.yaml:
```yaml
listener:
host: 127.0.0.1
port: 5456
postgres_replication:
postgres_server_address: 127.0.0.11
postgres_user: postgres1
postgres_password: password1
postgres_database: postgres1
postgres_port: 54322
slot_name: doltgres_slot_test2
behavior:
dolt_transaction_commit: true
```
logs of running `doltgres -config config.yaml`
```log
Starting replication
INFO[0000] Server ready. Accepting connections.
WARN[0000] secure_file_priv is set to "", which is insecure.
WARN[0000] Any user with GRANT FILE privileges will be able to read any file which the sql-server process can read.
WARN[0000] Please consider restarting the server with secure_file_priv set to a safe (or non-existent) directory.
```
It's supposed to throw error because the `postgres_server_address` and its info is incorrect, as well as slot name. In Ubuntu it's able to throw connection error if doltgres has trouble connecting. I set the config to incorrect info for troubleshooting after running the cli but no similar logs output found comparing to the ubuntu's.
Contributor guide
Assessment
This issue has not been assessed yet.