cockroachdb / cockroachdb/cockroach

Backup error

Open
#133,527 5 comments 0 reactions 0 assignees View on GitHub
A-disaster-recovery A-jobs branch-release-23.1 C-bug O-community P-3 T-jobs X-blathers-triaged
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

Hello , we have an issue in our production cockroachDB.
When i try to backup a table from our Database i run into this issue:
Node liveness error
![Node liveness](https://github.com/user-attachments/assets/d68c63cd-e13a-4eeb-ab29-55f7486824f4)
This is the command that i use:
`BACKUP public.message INTO 'nodelocal://1/1' AS OF SYSTEM TIME '-10s';`

BTW: our daily backups fail as well , this is the Command we use to schedule our daily backups:
```
CREATE SCHEDULE bkp_schedule FOR BACKUP INTO 'nodelocal://1/dailybackups' RECURRING '0 7 * * *'
FULL BACKUP ALWAYS
WITH SCHEDULE OPTIONS first_run = 'now';
```
*******
Our setup:
CPU

Intel(R) Xeon(R) CPU E5-2697 v4 @ 2.30GHz

Base speed: 2.30 GHz
Sockets: 18
Virtual processors: 36
Virtual machine: Yes
L1 cache: N/A

Utilization 21%
Speed 2.30 GHz
Up time 5:05:47:36
Processes 138
Threads 13931
Handles 135384

Memory : 160 GB
Windows :10 - 64 bit architecture
CockroachDB Version : 23.1.8
Running Command:
`%~dp0cockroach.exe start-single-node --insecure --cache=.25 --max-sql-memory=.25 && (echo 'didn't fail') || (start cockroach.bat)`
********
Also we are using CockroachDB for the purpose of Nakama chat and messaging System.
Our backup has been recently running into this issue.
I can't tell you how to reproduce this issue specifically because we have around 1 million+ users and millions of rows in the messages table of our database , but i can provide logs if necessary.
Thank you in advance

Jira issue: CRDB-43662

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.