12

I am using ZFS on Linux, and trying to setup remote replication. But I have bad ssh connection, and zfs send/receive over ssh restarts again. I know there is a issue on ZoL, but I don't know when it will be implemented, or a new stable release come. I've hear about mbuffer, but seems it can't do restart. Maybe it is possible to use it somehow like zfs send | mbuffer | while true; do ssh ...; done, but not sure.

Right now I am doing zfs send to a file, rsync it with --append --partial to remote and restore. but it consumes space, need manual assistance, and is a dirty solution. I want to have some automated solution, like sanoid/syncoid, to keep a mirror of my pool with all snapshots. Maybe some bash script that doing the same, but keeps all snapshots on remote, deletes files on host when they are successfully rsynced, etc. Please help

P.S. I know there is a duplicate question, but there is no any solution. In my question I use some dirty solution, and want to replace or improve it

Ural
  • 486
  • 1
  • 5
  • 8
  • 2
    [It's already implemented.](https://github.com/illumos/illumos-gate/commit/9c3fd1216fa7fb02cfbc78a2518a686d54b48ab8) Wait until it's available in your distribution of choice. Till then utilising rsync and a local file is the best you can get, I suppose. – Marco Feb 09 '17 at 10:51

1 Answers1

13

You can use the -s option of zfs receive which will save a resumable token on the receiving side if the transfer fails. It depends if you are using netcat (nc) or SSH.

On the recv machine (netcat only):

nc -l <port> | zfs receive -s -v tank/dataset

On the send machine:

Start with the usually send:

zfs send -v snapshot | nc <host> <port>

zfs send -v snapshot | ssh ... zfs receive -s -v tank/dataset

If the transfer fails, go on the recv machine and type:

zfs get all tank/dataset

Get the receive_resume_token and go on the send machine:

zfs send -v -t <token> | nc <host> <port>

zfs send -v -t <token> | ssh ... zfs receive -s -v tank/dataset

Here you go :)

Phil M.
  • 231
  • 2
  • 4
  • I know that non-linux zfs supports it for long time, I am still waiting for a new stable 0.7.0 release (with hopefully trim support), I am afraid to use pre-release in production (latest stable still don't support it) – Ural May 26 '17 at 04:17
  • 3
    Some additional notes: the token is changed after each object (snapshot in most cases) is transmitted, and destroyed after the whole file system has been fully transmitted. As it is saved for each file system, recursive send/recv does not play nice with it - it might be easier to just manually iterate over all file systems and transmit/resume them separately instead of using the built-in recursive option. – user121391 Nov 27 '17 at 15:33
  • How to update dataset in case of it changes on source to target? – Ozgur Erdogan Dec 19 '22 at 17:48