I’ve recently performed two conversions of physical hosts to Xen guests running on CentOS 5.1 host. The physical hosts were running RHEL 3.7 and CentOS 4.6. It is important to know that after the conversion you end up with a fully virtualized guest – HVM. This has its effect on the performance, especially when it comes to I/O. It is a general limitation of fully virtualized Xen guests and not of this particular P2V method. I don’t recommend using it for virtualizing for example a file server. In such a case your only real choice is to create a new paravirtualized guest and migrate the services. This article won’t be describing that approach.

These instructions are based on the article by Kyle Rankin. I’ve adapted it for Xen and added comments on those things which did cause me trouble. The instructions have been used to virtualize RHEL and CentOS servers to run as Xen guests on CentOS 5.1 but you should be able to use most of them for other distributions.

First step in the conversion is to create LVM volume(s) in the Xen host to be used by the guest. To figure out how much space you need, have a look at the physical server you want to virtualize. Each LVM volume will be visible in the guest as a separate hard disk. Note that HVM guests support only up to four disks. You will partition the disks from inside the guest in step two.

Second step is to create a new HVM guest and boot it from a bootable rescue CD/DVD. You should use a rescue CD which matches the architecture of the physical host being virtualized. In step four you need to run chroot command on the guest and it won’t work if you try for example a 32 bit rescue CD on a 64 bit system. After you have already booted the guest from a rescue CD you need to create partitions to match the physical host. The easiest thing is to match exactly the number and sizes of partitions on the physical system. You can also put several file systems which were on separate partitions on the physical server on a single partition on the guest as long as you prepare enough space.

Third step is to copy files from the physical server to the guest. This can take some time, so depending on whether you want to minimize the downtime you can do it either in one step or in two. Both methods use rsync command. The second method is described by Kyle in his article. The first method is basically the same but you only do the final synchronization. If you use a single transfer method you don’t need to use the --delete parameter for the final (and in this case the only) synchronization. The single transfer method is faster but it might not be available for you if you need to minimize the downtime.

Fourth step is to adjust the boot settings in the guest. Without that the guest won’t boot. To do that make sure all the guest file systems are mounted and chroot into the guest root file system. From there you need to adjust the boot loader configuration. I use GRUB on my systems. For instructions on adjusting LILO you’ll have to look elsewhere.

First you need to change the file /boot/grub/device.map. If you see /dev/sda there you need to replace it with /dev/hda. This should represent your boot drive. Also verify that the guest boot partition is in /etc/mtab. After that you execute grub-install /dev/hda.

Another file which needs modifications is /etc/grub.conf. Pay attention to the lines starting with boot= and splashimage= as they probably need adjusting. The boot parameters need to be updated as appropriate. The important entries are root (hd0,0) and root= kernel parameter. Note that the first one selects the boot partition while the second selects the partition for the root file system. So for example if the boot partition is in /dev/hda3 and root partition is /dev/hda1 these entries should look root (hd0,2) (here the partitions are counted starting from zero) and root=/dev/hda1 (here they are counted from one).

Fifth step is to generate a new initrd file for the guest. First copy the existing initrd file somewhere in case you need it later. Then adjust the /etc/modules.conf or /etc/modprobe.conf file (only one will be present depending on the distribution and OS version). Remove all entries starting with alias scsi_hostadapter. After that you can run mkinitrd command. I recommend you use -v parameter for mkinitrd. This will list all the modules which will be included in the initrd. You will probably need at least libata and ata_piix – even if the physical server uses SCSI disks the HVM guest will use IDE. This depends on the OS of the physical machine you’re virtualizing because it was only important when I was converting the RHEL 3.7 server but not with CentOS 4.6. If they’re missing you can add them using --with= parameter. Another advice in case you need to verify the content of initrd file (probably because the guest doesn’t boot): take into account that older versions of mkinitrd produce just a gzipped ext2 filesystem while the newer ones create a gzipped cpio archive.

Sixth step is to adjust the network settings and mount points for the guest. The configuration files you need to edit to change network settings are in /etc/sysconfig/network-scripts. Most likely you will have at least ifcfg-eth0. RHEL and CentOS usually specify the MAC address of the network adapter there so you need to change it to match with the MAC address of your guest. Otherwise when the guest boots the interface might not get activated. If you want to change the IP address of the guest you can also do it although I’d recommend to leave it for later when you already know the guest works. For the initial boot it should be easier if the guest runs with the same IP address as was used before by the physical server.

You also need to edit file /etc/fstab to match with the partition scheme defined for the guest.

Seventh step is to boot the HVM guest which will be now able to replace the physical server. In order to avoid conflicts you should either power down the physical server, disconnect it from the network or change its IP address(es). After that you can reboot the Xen guest. If you followed the instructions and if the initrd file contains all required modules the system should boot and start all the configured services. If you’re using kudzu it will ask you a few questions about removed/added hardware. Kyle suggests to select “Keep Configuration” for any removed SCSI or network hardware, and select “Ignore” for any added SCSI or network hardware. If you are prompted about any removed video, sound, USB and similar hardware you can safely select “Remove Configuration”. When booting finishes verify that all required service are running and that you have network connectivity from within the guest.

That’s it. You should have now a fully virtualized guest running on the Xen host which has the same configuration as the physical server it replaces. It would be a good idea to perform this process first on a test physical server so you have more time to figure out solutions to any problems you may encounter.

Related post(s)

• Xen and CentOS 5.1 on PowerEdge 2950 (2)
• USB forwarding on Xen – it just does not work (6)
• USB forwarding on Xen (1)
• What files did I change? (2)
• Setting up UPS on CentOS 5.2 with SELinux, part 2 (1)

Unfortunately the Xen guest was not stable. It would crash after half an hour, sometimes after two hours. I tried increasing the amount of RAM allocated to the guest, tried both paravirtualized and fully virtualized guests and the result was the same. The guest would crash.

I tried again to use USB forwarding and pass only a single USB device to the guest. After a few more attempts I noticed that I could “partially” forward the USB device itself to a fully virtualized guest (HVM) using this syntax:

usb=1
usbdevice="host:xxxx:yyyy"

The values for xxxx and yyyy are the vendor ID and product ID for the USB device. They can be found out by using the lsusb command. For example on my laptop lsusb produces this output:

Bus 007 Device 001: ID 0000:0000
Bus 005 Device 003: ID 0b97:7772 O2 Micro, Inc.
Bus 005 Device 002: ID 0b97:7761 O2 Micro, Inc.
Bus 005 Device 001: ID 0000:0000
Bus 004 Device 009: ID 062a:0000 Creative Labs
Bus 004 Device 008: ID 413c:2002 Dell Computer Corp. SK-8125 Keyboard
Bus 004 Device 007: ID 413c:1002 Dell Computer Corp. Keyboard Hub
Bus 004 Device 001: ID 0000:0000
Bus 003 Device 001: ID 0000:0000
Bus 006 Device 001: ID 0000:0000
Bus 002 Device 001: ID 0000:0000
Bus 001 Device 001: ID 0000:0000

So if I wanted to forward the device Dell Computer Corp. SK-8125 Keyboard I’d put the following in the Xen guest configuration file:

usb=1
usbdevice="host:413c:2002"


Ok, so in theory this should have let me see the USB device in Xen guest. Well it did, kind of. One thing I noticed was that I was only able to see some output when running lsusb on the guest when the guest was started from command line using the xm create command. When starting the guest from Virtual Machine Manager GUI I’d see absolutely no USB devices in the guest. Second, even when starting the guest from command line I’d see this kind of output from lsusb:

Bus 001 Device 001: ID 0000:0000

So it looks like something was being forwarded to the guest but it wasn’t the device I wanted to export. It looks like USB pass-through works on Xen only with some USB devices. That matches with what I’ve read at xgu.ru.

The final result – at the moment I’m using the USB device from the physical host. At least until I have time to look at it again.

Related post(s)

• USB forwarding on Xen (1)
• Xen p2v conversion in seven simple steps (1)
• USB in Xen – how to make it work (0)
• Xen and CentOS 5.1 on PowerEdge 2950 (2)
• How to Xenify office servers (0)