KVM - live external snapshots and appropriate disk names

Question

If I wish to take an external snapshot of a running KVM guest, I would run a command like below:

# create the directory for the snapshot data
mkdir /vms/$DOMAIN/snapshots/$SNAPSHOT_NAME

# create the snapshot
virsh snapshot-create-as \
--domain $DOMAIN $SNAPSHOT_NAME \
--diskspec vda,file=/vms/$DOMAIN/snapshots/$SNAPSHOT_NAME/disk.qcow2,snapshot=external \
--memspec file=/vms/$DOMAIN/snapshots/$SNAPSHOT_NAME/mem,snapshot=external \
--atomic

If my VM was called vm1 and my snapshot was called base-installation for taking a snapshot right after a fresh install, then I might have a folder structure like so:

/
|- vms/
   |- vm1/
      |- disk.qcow2
      |- snapshots/
           |-base-installation/
               |-disk.qcow2
               |-mem

Now this structure is what I want it to look like, but unfortunately the disk.qcow2 files are the wrong way round (need to be swapped). The disk.qcow2 file within the base-installation snapshot folder is actually the overlay disk image that takes all future writes to the VM, rather than being the desired read-only backing file representing the disk state at the time the snapshot was taken. The disk.qcow2 file directly under the vm1 folder is the read-only backing file, rather than the latest overlay disk image.

Question

Is there a way I can change the virsh snapshot-create-as command so it will store the read-only backing file to the specified disk-spec path, and keep the overlay pointer disk file where it already is? Alternatively, is there a workaround, like "pausing" the VM, swap the disk files, update the guest's xml definition, and then resume the VM? Ideally I don't want to have to shut down the guest or cause a noticeable period of downtime.

score 2 · Answer 1 · answered Mar 13 '17 at 23:54

I managed to find a workaround by performing an intermediary "snapshot" and a blockpull into that snapshot before deleting that from the metadata and performing the "real" snapshot, resulting in the overlay image being back where we started, and the data being in the snapshot folder. This is a bit of a hassle and hopefully there will be a better native solution in future, however, I have detailed the steps below, if like me you just can't wait.

Steps

First set your settings appropriately:

# Your settings may vary
DOMAIN="guest1"
SNAPSHOT_NAME="base-installation"
VMS_DIR="/vms"

Now we create the directory where we will store the snapshot data.

mkdir -p $VMS_DIR/$DOMAIN/snapshots/$SNAPSHOT_NAME

Now we create our intermediary snapshot which will create an overlay image in the snapshot directory we just created.

virsh snapshot-create-as \
--domain $DOMAIN intermediary_snapshot \
--diskspec vda,file=$VMS_DIR/$DOMAIN/snapshots/$SNAPSHOT_NAME/disk.qcow2,snapshot=external \
--memspec file=$VMS_DIR/$DOMAIN/snapshots/$SNAPSHOT_NAME/mem,snapshot=external \
--atomic

Note: I had ping running in the VM and it appeared to pause for only a second when that command ran.

Now we use blockpull to move the data from the backing file into the active overlay image in the snapshot directory.

virsh blockpull \
--domain $DOMAIN \
--path $VMS_DIR/$DOMAIN/snapshots/$SNAPSHOT_NAME/disk.qcow2 \
--wait \
--verbose

Note: You must not provide the --base argument for that command, otherwise it will do nothing because if you specify --base it makes the command leave the specified base as the base directory and will only merge the intermediary layers (which we have none of) and not the base directory itself.

The previous command took me about 30 seconds, but the vm was happily running ping during that time.

ls -alh shows that the snapshot directory's disk.qcow2 image is now a healthy 2.2GB in size, but more importantly, qemu-img info disk.qcow2 shows no backing file now, so we know all the data is now in the snapshot directory and it is a "standalone image".

Now we clean up the metadata by removing knowledge of this intermediary snapshot step.

virsh snapshot-delete \
--domain $DOMAIN \
intermediary_snapshot \
--metadata

Now we also remove the original backing file that we no longer need, as well as the intermediary memspec file which we will never use.

rm $VMS_DIR/$DOMAIN/disk.qcow2
rm $VMS_DIR/$DOMAIN/snapshots/$SNAPSHOT_NAME/mem

At this point I tested shutting down and then manually starting the instance to check it was happy and everything works.

At this point you can think of having finished all the "preparation" now we simply take the snapshot but are specifying the overlay image goes in the top level $DOMAIN directory, and the memspec file goes in the snapshot folder beside it's backing disk image.

virsh snapshot-create-as \
--domain $DOMAIN $SNAPSHOT_NAME \
--diskspec vda,file=$VMS_DIR/$DOMAIN/disk.qcow2,snapshot=external \
--memspec file=$VMS_DIR/$DOMAIN/snapshots/$SNAPSHOT_NAME/mem,snapshot=external \
--atomic

That's it!

Storage Impact

It's worth noting that if you were to keep repeating the steps above to make many snapshots, you would end up with fully restorable snapshots that don't depend on any other files in each snapshot folder, but this will waste a lot of space. For subsequent snapshots, you would probably want to alter the blockpull command with the --base argument, to have each snapshot use the previous snapshot as a backing image, and not merge it.

Restoring The Snapshot

If you perform a snapshot-list you see your new snapshot. However you will not be able to perform the virsh snapshot-revert command as reverting external snapshots is not supported yet (at least not in ubuntu 16.04 that I'm using) so we need to do this manually by destroying the current active overlay image....

rm $VMS_DIR/$DOMAIN/disk.qcow2

Then create a new overlay image from the snapshot base image we wish to restore from:

qemu-img create \
-b $VMS_DIR/$DOMAIN/snapshots/$SNAPSHOT_NAME/disk.qcow2 \
-f qcow2 \
$VMS_DIR/$DOMAIN/disk.qcow2

Now you can just start up the guest.

sudo virsh start $DOMAIN

I haven't yet figured out how to use the memspec files unfortunately and is the biggest thing missing from this answer.

References

In trying to figure this all out, there was a great post fedorapeople.org that explains how to merge external snapshots that I found most helpful.