3

I have a zpool with around 6TB of data on it (including snapshots for the child datasets). For the important data in it, I already have backups on filesystem level. As I need to perform some rather "dangerous" operations (i.e. this pool migration), I want to backup the whole pool to a different server.

In an ideal world, I would have used send and recv (like so). Unfortunately, this server is btrfs-based with no option to install zfs. When researching, people recommend just plain rsync (e.g. here) but as far as I can see, I would be back to filesystem level and more importantly, I'm not sure if the already existing dataset snapshots would still be intact. I basically just want to "freeze" the entire pool with a snapshot, send that off to the remote server and in case something goes wrong, restore the pool to the previous state (with ideally one command).

Therefore I'm looking for a solution to backup an entire zfs to a different server running a different filesystem, keeping all datasets and snapshots intact

Dom42
  • 31
  • 2
  • When you mentioned btrfs, you mean the designated backup server was using btrfs rather than zfs, right? You may still send the file system to a file on the destination server... – Kusalananda Aug 29 '22 at 09:36
  • Exactly, the backup-server uses btrfs as a filesystem. The volumes cannot be changed to zfs and also no zpool can be created on the backup server. With the second part of the comment, you refer to just creating a snapshot of the pool and then transfer that snaphot over to the second server? Will this keep all child filesystems (including child snaphots intact)? – Dom42 Aug 29 '22 at 09:39
  • 1
    I'm consciously doing a bit of handwaving here because I don't feel I have the in-depth zfs experience needed to speak with any kind of authority, but I do know that you can send to a file, so you would not actually need to have a zfs filesystem waiting on the receiving end, just something big enough to store the filesystem as a file. Sending can, AFAIK, only be done on snapshots, but sending with `-R` (`--replicate`) will also send descendent filesystems. – Kusalananda Aug 29 '22 at 09:48
  • Yes, sending to file seems to be correct (e.g. also mentioned here: https://unix.stackexchange.com/a/113752/539348). However, I was wondering if there might be a different solution and/or perform a sanity check on my plan. – Dom42 Aug 29 '22 at 09:57
  • 1
    When you say "*and also no zpool can be created on the backup server*", is that because there are no unused drives/partitions/block-devices on the backup server, or because the zfs module & zfs utils are not installed on it? A zpool can be created using one or more files instead of disks/partitions (just create the files first. see `man zpool`, search for Example 4). Alternatively, you can create a VM that has a zpool made from one or more virtual disks. – cas Aug 30 '22 at 00:07
  • 2
    But if all you want is a temporary full backup while rebuilding the system, `zfs send`-ing to a file is good enough (it only becomes a pain if you also need to keep incremental backups and expire them as they get older...as in the Q you linked to). For just a one-off backup, you can save a full `zfs send -R` to a file on any kind of filesystem, and restore from it with `zfs recv` later. It's just a stream of data. – cas Aug 30 '22 at 00:09
  • Or *files* rather than a single *file* - most filesystems have file size limits. Use `split` in the pipeline (on the receiving system) to split your 6TB backup into, say, 4GB chunks. You should probably also compress the send (on the sending system, to minimise network traffic) with `gzip` or `xz` or something, unless most of the data is already compressed (e.g. video files). – cas Aug 30 '22 at 00:22
  • @cas Thank you for weighing in on this! What I meant by "no zpool can be created" is that the destination system does not have `zfs` (module/utils) available and also no VM can be installed there. And yes, I only need to have one non-incremental backup. (As the pool migration would actually always leave two copies on temporarily non-redundant pools even without the additional backup, I should be fine anyways, but better safe than sorry). Do you feel like fomulating a whole answer? – Dom42 Aug 30 '22 at 18:50
  • 1
    @cas [`pigz` is much better in this case](https://zlib.net/pigz/) - parallelizing the compression will improve transfer speed considerably. As will avoiding an `ssh` connection - if it's safe to `rsh`, that'll be a lot faster, too. Also, use `mbuffer` in the pipeline as `zfs send ...` tends to send data in large bursts with relatively long idle periods between the bursts: `zfs send ... | mbuffer ... | pigz ... | mbuffer ... | rsh someone@somehost split ...` – Andrew Henle Aug 31 '22 at 17:24

0 Answers0