March 2010 Archives

I wrote about how to arrange disks on a host that needs to support RH-style virtual machines (VMs) in a flexible way.

And what should you do on the guest (the VM itself)?

Well, following the cited scheme, the VM will be handed an LVM "logical volume" to use.

The VM will treat those bits as a disk (as it would if you created a big file and handed that over, instead). Your only real choice is to partition that virtual disk, and use the resulting partitions in the usual ways.

If you know exactly how to use the VM's disk space, and "it will never change", then create fixed partition(s), slap filesystem(s) on them, and you're done.

If, however, we again assume a need for "flexibility", then LVM is again a good idea -- yes, LVM(guest)-within-LVM(host).

I have been told (but not tried myself) that LVM on the host can "see" the inner-LVM stuff (the guest's) and do things with it, e.g. snapshot it.

(Apparently important: the volume group(s) on the guest must have different names from the volume group(s) on the host.)

But the main virtue of LVM-within-LVM is that it's relatively simple to increase the "disk space" on the guest...

Stop the VM: virsh shutdown guest

On the host, extend the logical volume that is the guest's "disk":

lvextend -L20G /dev/vg0/guest-disk

Restart the VM: virsh start guest

The guest's "disk" just got magically bigger. Use fdisk (or equiv) to make a new "partition" out of the extra space, giving it the LVM type (8e).

Make the new "partition" an LVM "physical volume" (PV):

pvcreate /dev/xvda3

Add the PV to the VM's "volume group" (VG):

vgextend guestvg0 /dev/xvda3

Now you can make one of the "inner" (guest) "logical volumes" (LVs) bigger:

lvextend /dev/guestvg0/lvroot ...

Finally, the filesystem sitting on that LV will need resizing:

resize2fs /dev/guestvg0/lvroot

If you're going to use a Linux box for Red Hat-style virtualization (Xen/KVM), how should you use/partition your disk(s)? Here's one answer.

First: disks are cheap enough that I always use two, and software-RAID-1 them together (mirroring). Plan to do that.

Second: Though I'm no fan of LVM (the Logical Volume Manager) -- extra complication for no gain -- it's a good choice for "host for a bunch of VMs (virtual machines)". I'm assuming you don't know exactly which VMs of what size, so the flexibility of LVM is actually useful.

Also, as disks get larger, it's harder to predict in advance how you'll end up using all that space. So, again, the flexibility of LVM may benefit.

Now, I don't use LVM for the basic filesystems (/boot and / [root]). Their sizes are predictable and, when It Goes Horribly Wrong (TM), you'll be really glad not to have LVM in the way. So I partition my raw disks thusly:

Filesystem  Size  Used for
/dev/md0    300M  /boot
/dev/md1     16G  /
swap         ???
/dev/md2    rest  of the disk(s)

That /dev/md2 goes straight into LVM. Make a PV (Physical Volume):

pvcreate /dev/md2

And create a Volume Group (let's call it vg0) with it:

vgcreate -s 32M vg0 /dev/md2

(That -s 32M is not particularly well-informed.)

Now you can make LVs (Logical Volumes) to your heart's content...

lvcreate -n vm001-disk --size 10G vg0

... and feed them to your VM-creating software. (Or make things for your host machine, e.g. a varying-sized /var.)

A word on LVM-on-RAID... LVM has a mirroring facility of its own, so why use software RAID1?

First, I'm already paying the software-RAID tax with my non-LVM partitions (/boot and /).

Second: For me, software RAID is familiar and has been reliable.

Third: Software RAID-1 is less scary in a crisis. I like things like: You can take a disk that is half of a RAID-1 mirror, throw it in an enclosure, and mount it as a filesystem, no problem.

Fourth: The LVM mirroring guff seemed more complicated and/or easier to get wrong. (Just unfamiliarity, maybe.)