digint / digint/btrbk

Failed send/receive with "inode_cache" mount option

Open
#253 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
Perl
Stars
2.1k
Forks
139
PR merge metrics
No merged PRs in 30d

Description

I hope this helps others. I don't see this as a bug with btrbk, but rather a brtfs problem with inode_cache and send/receive. I was using btrbk for months without issue to send/receive for backups. Then I had a power blip and a set of other issues that corrupted a RAID0 pair of drives. I went back to the backups on the USB drive and discovered that there was a failure in my backups that I didn't detect. I was able to recover much of the drive content after setting up a new btrfs file system.

But when I began to back up the new file system, I was experiencing a successful first, complete send to populate the backup device, however all the subsequent incremental sends were failing:

`2018-10-07T19:35:45-0500 send-receive starting /media/jpb/btr-backup/@.20181006 /mnt/root-home/btrbk_snapshots/@.20181006 /mnt/root-home/btrbk_snapshots/@.20181005 -
2018-10-07T19:35:47-0500 send-receive ERROR /media/jpb/btr-backup/@.20181006 /mnt/root-home/btrbk_snapshots/@.20181006 /mnt/root-home/btrbk_snapshots/@.20181005 -
2018-10-07T19:35:47-0500 delete_garbled starting /media/jpb/btr-backup/@.20181006 - - -
`

I was experiencing other snapshots that were successful against my newly created file system but not against one that had survived the power blip. So, I was scratching my head, looking for why btrbk was working for some, but not all on the same system.

The error message when trying to run the send/receive manually was a file not found that began with an o followed by some numbers. I don't have a good example. Essentially, it was an orphan file being presented during the transfer. Send/receive was having trouble with these files. It would find it on one side of the transfer but not the other.

Then I remembered I had turned on inode_cache as a mount option. I had not reactivated it on my newly created repaired drives.

Changing /etc/fstab from:

`Caused the send/receive to fail:
UUID=f7dca671-def7-49bf-a147-9f3db364bfdd / btrfs defaults,subvol=@,autodefrag,space_cache,noatime,inode_cache 0 1
`

To:

`Fixed the send/receive:
UUID=f7dca671-def7-49bf-a147-9f3db364bfdd / btrfs defaults,subvol=@,autodefrag,space_cache,noatime 0 1
`

I then rebooted, deleted all the btrbk snapshots for this subvolume, manually created the initial snapshot just as btrbk would and then issued a btrbk resume. The system has been running perfectly since.

I had added inode_cache to attempt to improve the performance against a pair of 4TB drives configured in btrfs RAID0 and another pair of 2TB drives configured in btrfs RAID1. I didn't notice a performance improvement, but did notice the send/receive failure -- just too late.

Contributor guide

No contributing guide indexed for this repository

Research direction

No source file or test is named. Start with the reported btrbk send/receive paths, the /etc/fstab mount options, and a manual incremental btrfs send/receive using inode_cache enabled and disabled. Done would require a maintainer-confirmed diagnosis and a clearly defined code or documentation change.

Written by the indexing model from the issue text.

Assessment

Tech stack
linux
Domain
operating-systems
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.