6

I have two separate btrfs filesystems which I would like to consolidate. The subvolume structure looks like this:

/ filesystemA
|- subvolume1
|- subvolume2
|- ...
|- subvolumeN

/ filesystemB
|- subvolume1
|- subvolume2
|- ...
|- subvolumeM

filesystemB is the target. I would like to move subvolumes 1 to N into filesystemB. N is more than 1000, as they are daily-snapshot subvolumes - also because of this the differences between them are minimal, i.e. there is a lot of shared data. filesystemA is 1TB, but a naive du on the path totals more than 300TB of files (actual space usage on the drive is less than 500GB).

Therefore I believe a straight mv or cp from filesystemA to filesystemB is infeasible, as it won't be aware of btrfs subvolumes or the shared nature of the data.

I've read the documentation on btrfs send and btrfs receive a couple of times and it looks like this might work, essentially rebuilding the whole subvolume structure on filesystemB incrementally. But I'm hoping there is a simpler way, as I'd have to script this (due to the large number of subvolumes) and I feel this may be error-prone.

(Additionally, send/receive requires the subvolumes to be read-only, and the version of btrfs-tools on Ubuntu 14.04 doesn't seem to have the btrfs property command that would allow me to set an existing subvolume to read-only. I could install a later version but would prefer not to for the sake of configuration management on this "production" machine.)

jl6
  • 1,415
  • 2
  • 13
  • 29
  • `btrfs send` and `btrfs receive` can be put into a simple script which should do the job. I did this last year for a few hundred subvolumes (daily/weekly snapshots of ~10 live subvolumes) – Mark K Cowan Nov 30 '16 at 13:00

2 Answers2

3

I am pretty sure, you won't get around btrfs-send/btrfs-receive. Another option would be, to copy the subvolumes and then deduplicate them on the destination, but this would take ages.

For the readonly problem: as these are daily snapshots, arent most of the volumes read-only anyway?

I would do a

$btrfs subvolume create /subvolumeA/source
$for i in /filesystemA/subvolume*; do btrfs subvolume snapshot -r $i /filesystemA/source/$i; done

Now you have all subvolumes under /filesystemA/source/ readonly.

Now you could send them in a loop incrementaly with btrfs-send with the parent statement to the destination.

If some of the snapshots should be writeable again, you can copy the content of these to a new writeable subvolume using cp --reflink.

Jodka Lemon
  • 3,143
  • 13
  • 42
2

I hope you got somewhere with your list. Are you sure the property commands aren't undocumented on that version? This answer suggests you can use a command like btrfs property set -ts /path/to/snapshot ro true to correct the issue.

For future readers: Set new snapshots to read-only, with the -r parameter when you create them. I'm using something like this in my script:

comm -23 \
    <(btrfs subvolume list -ors --sort=path /fsA/ | perl -ne 'print if s/.+?\spath\s+(.+)/\1/') \
    <(btrfs subvolume list -ors --sort=path /fsB/ | perl -ne 'print if s/.+?\spath\s+(.+)/\1/')

This should provide a list of read-only snapshot volumes on file-system A which are not on file-system B. You can iterate the list, doing send/receive on each one to synchronise.

Walf
  • 1,254
  • 1
  • 14
  • 19