Step by step guide to setting up Xen and XAPI (XenAPI) on Ubuntu 12.04 and managing it with Cirtix XenCenter

XCP ( Xen Cloud Platform ) is the open source version similar to Citrix XenServer that uses the Xen Hypervisor. It is currently distributed as an ISO installer also called as XCP appliance. XCP uses XAPI or XenAPI to manage Xen hosts. XCP is based on CentOS 5.5

Project Kronos is an initiative to port the XAPI tool stack to Debian and Ubuntu. It is a management stack implemented in OCaml that configures and controls Xen hosts, attached storage, networking and virtual machine life cycle. It exposes a HTTP API and provides a command line interface (xe) for resource management.

XenCenter is windows desktop application by Citrix that is distributed with XenServer for managing servers running XenServer. It uses XAPI for talking to Xen resource pools. Since we are setting up XAPI, we can use XenCenter to manage the server

Why use XCP-XAPI on Debian/Ubuntu when XCP appliance exists ?

  1. Manage dom0 using a configuration management framework (Puppet, Chef)
  2. Apply security updates to dom0 root file system
  3. Run Xen version 4.1
  4. Ubuntu 12.04 is a LTS release that is supported for 5 years

Prerequisites

  • A fresh installation of Ubuntu 12.04 on the server
  • Small root file system partition – I usually have a 10GB partition for root fs (/) and the rest of the space is setup as a physical volume for setting up LVM later. This LVM partition will be used for vm storage and snapshots later. You can choose any partition layout that you are comfortable with, just remember to keep the root partition small and have a large space dedicated for a LVM volume
  • root access to the host

Installing and configuring Xen Hypervisor

# Install the Xen Hypervisor
 $sudo apt-get install xen-hypervisor

# Setup GRUB to boot the Xen Hypervisor
$sudo sed -i ‘s/GRUB_DEFAULT=.*+/GRUB_DEFAULT=”Xen 4.1-amd64″/’ /etc/default/grub

# Disable apparmor at boot
$sudo sed -i ‘s/GRUB_CMDLINE_LINUX=.*+/GRUB_CMDLINE_LINUX=”apparmor=0″/’ /etc/default/grub

# Restrict dom0 to 2GB of memory and 2 vcps
$sudo vi /etc/default/grub
after GRUB_CMDLINE_LINUX=”apparmor=0″  add the line GRUB_CMDLINE_XEN=”dom0_mem=2G,max:2G dom0_max_vcpus=2″

# Update Grub with the config changes we just made
$sudo  update-grub

# Reboot the server so that Xen boots on the server
$sudo reboot

#Once the server is back online ensure that Xen is running
$cat /proc/xen/capabilities should display “control_d”

Installing and configuring XAPI (XenAPI)

# Install XCP-XAPI
$ sudo apt-get install xcp-xapi 
 - choose bridge when prompted for network backend

# Setup the default toolstack
$ sudo vi /etc/default/xen – set ‘TOOLSTACK=xapi’

#Disable xend from starting at boot
$ sudo sed -i -e ‘s/xend_start$/#xend_start/’ -e ‘s/xend_stop$/xend_stop/’ /etc/init.d/xend

# NOTE: only xend the deamon needs to be disabled from starting, /etc/init.d/xend handles other things like modules and xenfs.  Do not disable it from the runlevel

# Disable service xendomains
$ sudo update-rc.d xendomains disable

# Fix for qemu which emulates the console does not have the keymaps in the correct location
$ sudo mkdir /usr/share/qemu; sudo  ln -s /usr/share/qemu-linaro/keymaps /usr/share/qemu/keymaps

# Setup bridge networking
$ sudo vi /etc/network/interfaces

# Create a bond called xenbr0. The file should look like this for static network configuration
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).
# The loopback network interface
auto lo
iface lo inet loopback# The primary network interface
auto xenbr0
iface xenbr0 inet static
address < eth0 ip address here >
netmask < eth0 netmask address here >
network < eth0 network address here >
broadcast < eth0 broadcast address here >
gateway
# dns-* options are implemented by the resolvconf package, if installed
dns-nameservers 4.2.2.2
bridge_ports eth0
iface eth0 inet manual
    
# Configure xcp to use bridge networking instead of openswitch
$ sudo vi /etc/xcp/network.conf
- replace “openswitch” with “bridge”

# All set – ready to reboot and let xcp-xapi toolstack take over
$ sudo reboot

# On restart – confirm that xcp is working
$ sudo xe vm-list
 - This should list the control domain
 uuid ( RO) : dbcf74d2-ee50-edd5-d44d-b81fc8ba1777
 name-label ( RW): Control domain on host: ubuntu-xenserver-1
 power-state ( RO): running

# If your output looks similar – xapi is running on the server, if you get “Connection refused”  then xapi is not setup correctly. Have you followed all the steps ?       

Setting up the LVM storage volume

Assuming that you configured a large partition for use as a LVM volume during installation, this section sets up the partition and adds it as a local storage repository

# verify that you have a LVM partition
$ sudo  fdisk -l
- This should list a partition of type “Linux LVM”. If you don’t see a partition and you have free space on the disk, create a new partition of type “Linux LVM” (8e)

# If you have partition of type “Linux LVM” follow the steps below create a physical volumes
$ sudo pvcreate  /dev/cciss/c0d0p2


$ sudo pvdisplay
# You should see similar output
“/dev/cciss/c0d0p2″ is a new physical volume of “947.60 GiB”
— NEW Physical volume —
PV Name /dev/cciss/c0d0p2
VG Name
PV Size 947.60 GiB
Allocatable NO
PE Size 0
Total PE 0
Free PE 0
Allocated PE 0
PV UUID rNeGnf-TbJS-vfSm-t7la-wNCv-Lpc3-vjn33c

# create a volume group
$ sudo vgcreate  VolumeGroup /dev/cciss/c0d0p2

# verify the newly created volume group
$ sudo pvdisplay 
 — Physical volume —
 PV Name /dev/cciss/c0d0p2
 VG Name VolumeGroup
 PV Size 947.60 GiB / not usable 2.90 MiB
 Allocatable yes
 PE Size 4.00 MiB
 Total PE 242584
 Free PE 242584
 Allocated PE 0
 PV UUID rNeGnf-TbJS-vfSm-t7la-wNCv-Lpc3-vjn33c
 
 # create a logical volume on “VolumeGroup”
 $ sudo lvcreate –size 947G  -n LocalStorage VolumeGroup
 
 # Verify the newly created logical volume
 $ sudo lvdisplay
 — Logical volume —
 LV Name /dev/VolumeGroup/LocalStorage
 VG Name VolumeGroup
 LV UUID pCWgAs-cpfh-IAdU-uVMi-EJbo-iy2x-TlMzar
 LV Write Access read/write
 LV Status available
 # open 0
 LV Size 947.00 GiB
 Current LE 242432
 Segments 1
 Allocation inherit
 Read ahead sectors auto
 - currently set to 256
 Block device 252:0
 
# Register the logical volume for use with XAPI
$ xe sr-create type=ext name-label=Local Storage device-config:device=/dev/mapper/VolumeGroup-LocalStorage 

– this will take a while if the volume is large

# Display the newly added storage
$sudo xe sr-list name-label=Local Storage  
 uuid ( RO) : 7dea0028-ee94-6c16-2f61-c699ed4a1d18
 name-label ( RW): Local Storage
 name-description ( RW):
 host ( RO): ubuntu-xenserver-1
 type ( RO): ext
 content-type ( RO):

Using XenCenter for managing our server

  1. Download and install XenCenter – instructions here
  2. start XenCenter and click Add New Server
  3. Put in the server’s ip username and password as setup during installation – let the magic begin

Known issues and workaround

  1. No console appears for dom0 in XenCenter

Additional resources

  1. Designing XenServer 6.0 Network Configurations