0

I'm running Arch Linux. I want to clone a 2 disk encrypted logical volume in a single volume group (LUKS on LVM). There is a slight catch. I want to swap some of the drives.

I have:

  • VG1: LV: PV(OldDrive1) + PV(OldDrive2)
sda (OldDrive1)
    -vg1-luks_encrypted_lv
sdb (OldDrive2)
    -vg1-luks_encrypted_lv

I have two other drives (NewDrive1 and NewDrive2). I want to create a VG2 that is a clone of VG1. However, I want to swap some drives around. So I want:

  • VG1: LV: PV(OldDrive1) + PV(NewDrive1)
  • VG2: LV: PV(OldDrive2) + PV(NewDrive2)
sda (OldDrive1)
    -vg1-luks_encrypted_lv
sdb (OldDrive2)
    -vg2-cloned_luks_encrypted_lv
sdc (NewDrive1)
    -vg1-luks_encrypted_lv
sdd (NewDrive2)
    -vg2-cloned_luks_encrypted_lv

My current plan is to clone each drive. I was previously thinking about using dd, but after some reading maybe I need to use pvmove?:

  • OldDrive1 -> NewDrive1
  • OldDrive2 -> NewDrive2

Could I then just swap the physical drives in LVM, because the drives are bit-by-bit clones? I'm worried I'm missing something. How would I incorporate the news drives into the LV? I would appreciate some advice, because I don't want to lose any data. Thanks.

Edit:

@telcoM 's answer worked very well. Thank you very much. I used the on-line method. If anyone wants to do something similar there are few things worth noting.

On Step 7:

The lvconvert -m default is now raid1, not lvm's own mirror system. Read man lvconvert for more details. Since I wanted to immediately split the mirror, it was much easier to just use the lvm's legacy mirror with the mirrorlog stored in memory:

lvconvert --type mirror -m +1 --mirrorlog core vg1/luks_encrypted_lv OldDrive2 NewDrive2

Just remember that --mirrorlog core puts the mirrorlog in memory. So don't turn off your computer before running lvconvert --splitmirrors or you will lose your mirrorlog file.

On Step 9:

Before you do vgsplit you need to unmount the filesystem and deactivate the logical volume.

On Step 11:

Most people probably realize this, but you need to assign a UUID to $uuid before you run cryptsetup luksUUID --uuid $(uuid) /dev/mapper/VG2-LVx. Run something like uuid=$(uuidgen) first.

Teatree
  • 93
  • 1
  • 5

2 Answers2

2

"Need" is a strong word - there is more than one way to achieve what you want.

With pvmove, you could do it on-line, while the encrypted LV is in use.

1.) pvcreate NewDrive1

2.) vgextend VG1 NewDrive1

3.) pvmove OldDrive2 (means effectively: "move any LVM-allocated extents from OldDrive2 to any other drive(s) in VG1 so that OldDrive2 becomes completely unallocated, if possible." This will take some time: you might want to run it within a screen/tmux session with a -verbose option.)

4.) Use pvs or pvdisplay OldDrive2 to make sure OldDrive2 is now completely unallocated.

5.) pvcreate NewDrive2

6.) vgextend VG1 NewDrive2

7.) for every LV in VG1: lvconvert -m +1 VG1/LVx OldDrive2 NewDrive2 ("create a mirror from VG1/LVx, allocating space for the mirror from OldDrive2 and NewDrive2"). If there is no space for on-disk mirror log, you might need to use --mirrorlog core option here.

8.) once mirrors are in sync, for every LV in VG1: lvconvert --splitmirrors 1 --name LVcopyx VG1/LVx OldDrive2 NewDrive2 ("Split off one mirror of LVx located on OldDrive2 and/or NewDrive2 and name it "LVcopyx" to avoid a name conflict.)

9.) vgsplit VG1 VG2 OldDrive2 NewDrive2 ("Split OldDrive2 and NewDrive2 off VG1, taking their LVs with them, and name the resulting new VG as VG2.")

10.) for every LV in VG2: lvrename VG2 LVcopyx LVx to restore the original LV name(s), now that the copies have been separated into their own VG and there is no more conflict. You now have a new VG2 that contains copies of VG1's LVs as they existed at the point of splitting the LV mirrors at step 8.).

11.) Before actually using VG2, you'll need vgchange -ay VG2, and then cryptsetup luksUUID --uuid $(uuid) /dev/mapper/VG2-LVx to give it an unique UUID distinct from its VG1 counterpart, and once you have unlocked the encryption, you should also give the filesystem inside it a new UUID too. For BtrFS, this is vital (btrfstune -u /dev/mapper/VG2-LVx-crypt); for other filesystems this is essentially just a convenience so that UUID-based mounting will work.

If you can have the VG off-line and can unplug/re-plug disks, you could also:

1.) Unmount, cryptsetup luksClose and de-activate the VG (vgchange -an VG1). To avoid unwanted auto-activation at boot or hot-plug time, also mark it as exported (vgexport VG1).

2.) Clone the drives as you planned.

3.) Unplug drives so that the system will only see OldDrive2 and NewDrive2. If your hardware allows doing it hot, use echo 1 > /sys/block/<device name>/device/delete for a graceful hot-unplug.

4.) Boot the system or run vgscan after a graceful hot-unplug. Then import and rename the VG: vgimport VG1, then vgrename VG1 VG2. Use vgchange --uuid VG2 to give the new VG2 a new identity distinct from the old VG1, and use pvchange --uuid OldDrive2 and pvchange --uuid NewDrive2 to do the same at the PV level. After importing and renaming, remember that you'll need to activate the VG before you can mount it or do any other operations to it: vgchange -ay VG2.

5.) After activating the VG, use cryptsetup luksUUID --uuid=$(uuid) /dev/mapper/VG2-LVx to do give the LUKS container a distinct new identity, and after unlocking the encryption, use a filesystem-specific tool to do the same at the filesystem level too. (this is especially important for BtrFS: btrfstune -u /dev/mapper/VG2-LVx-crypt)

6.) Now you can plug OldDrive1 and NewDrive1 back in (use vgscan if you hot-plug), and vgimport VG1 "again", and activate it with vgchange -ay VG1. You now have two fully separate VGs you can use as you see fit.

telcoM
  • 87,318
  • 3
  • 112
  • 232
  • Will these steps make the new pv (NewDrive1) encrypted too? [This](https://unix.stackexchange.com/questions/618877/how-can-i-add-a-new-physical-volume-to-extend-an-existing-luks-encrypted-lvm-vo) makes me think maybe I need to run _cryptsetup luksFormat_ on NewDrive1 run before I do pvmove. – Teatree Oct 03 '21 at 01:15
  • You said you have **LUKS on LVM**, so `lsblk` would show `part` -> `lvm` -> `crypt`. That means the each LV would be independently encrypted, and LVM operations would move or copy the encrypted data. The OP of the question you linked has **LVM on LUKS** (`part` -> `crypt` -> `lvm`), so his LVM operates on the unencrypted data and he needs each LVM PV independently encrypted if he wants to keep the volume group fully encrypted. If you are uncertain, please edit your original post to show your `lsblk` listing. – telcoM Oct 03 '21 at 08:52
  • I do have LUKS on LVM, but I wanted to double-check. Thanks for the clarification. I added more information to my OP anyway to help others that might have a similar issue. – Teatree Oct 04 '21 at 01:51
  • Sorry for dragging this out, but after the command on Step 7 and I get the message: ```"Are you sure you want to convert linear LV VG1/LVx to raid1 with 2 images enhancing resilience?"``` I read that LVM has switched to raid1 as default for ```lvconvert -m```. Do I want to use raid1 or do I want to use the legacy lvm mirror? Does it matter? Thanks. – Teatree Nov 04 '21 at 01:33
  • Raid1 is actually better than the old mirror, in that it automatically takes care of the mirrorlog placement. But if that fails because you have no space for the mirror log, then the old-style mirror with `--mirrorlog core` would be a way to work around it. Since you're going to split the mirror as soon as it finishes anyway, using a RAM-based mirror log should be tolerable. – telcoM Nov 07 '21 at 09:44
0

if you are able to close the LUKS container during the dd clonig, the steps could be as follows:

  • add NewDrive1 to VG1
  • use "pvmove OldDrive2 NewDrive1"
  • remove OldDrive2 from VG1
  • create VG2 using OldDrive2 + NewDrive2
  • create your desired LV on VG2
  • use dd VG1/LV -> VG2/LV

if you plan to use both copies on the same machine you will also have to change the UUID of the LUKS container and the containing filesystem! thanks to @telcoM for pointing that out!

brt
  • 113
  • 3
  • This would work too, but one should have the encrypted LV unmounted and closed while `dd`ing. Also, you probably should give a new unique UUID to the cloned LUKS container and the filesystem inside it (if the filesystem is BtrFS, you should **definitely** do it if there is a slightest chance you'll be mounting both copies within the same system at any future time). – telcoM Sep 19 '21 at 12:21
  • you are absolutely right! using `dd` on an active filesystem / LUKS container is never a good idea, also changing the UUID is important if you want to use both copies on the same machine. thanks for the hints i will add it to my answer. i just wanted to give a simple overview of the needed steps assuming that the user is knowing what he is doing ;) - i really like your answer using LVM mirroring as it can be done on-line. – brt Sep 20 '21 at 12:46