Personal tools
You are here: Home → Linux → Xen → CentOS → Workaround for CentOS 7 Xen VM with grub2 and Xen dom0 with pv-grub
 

Workaround for CentOS 7 Xen VM with grub2 and Xen dom0 with pv-grub

Note: This workaround is no longer required. The base Xen hypervisor and pacakges in Fedora 26 provides this functionality out of the box. The log file '/var/log/xeb/bootloader.<domain>.log' shows:

Using <class 'grub.GrubConf.Grub2ConfigFile'> to parse /boot/grub2/grub.cfg


Disclaimer: These are notes that relate to the residual for getting a CentOS 7 (el7) Xen VM to boot cleanly using it's installed kernel. In theory the dom0 machine should use a pv-grub2 binary to read the '/boot/grub2/grub.cfg' to select the correct kernel/initrd. What I have found is that Fedora 21 doesn't appear to have pv-grub2 support (as of December 2014). The pv-grub support is v1 of grub.

Workaround

Manually create and maintain in the domU VM a '/boot/grub/menu.lst' file. Configure the xl.conf file to use pv-grub and the menu.lst configuration file. When new kernels are installed in the VM update the menu.lst (groan).

 

Menu.lst

The following file should be put in the '/boot' filesystem as '/grub/menu.lst' (in this case '/boot/grub/menu.lst')

Notes:

  1. Given this is a VM this example has the /boot filesystem as part of the root filesystem (so the '/boot' prefix is required')
  2. The root filesystem is on the first block device (i.e. 'hd0'), in the first partiton partition table (i.e. ',0') part of the root.
default=0
timeout=5

title=CentOS 7 vmlinuz-3.10.0-123.13.2.el7.x86_64
    root (hd0,0)
    kernel /boot/vmlinuz-3.10.0-123.13.2.el7.x86_64 root=LABEL=/ ro vconsole.keymap=us
        console=hvc0 selinux=0 crashkernel=auto vconsole.font=latarcyrheb-sun16
    initrd /boot/initramfs-3.10.0-123.13.2.el7.x86_64.img

xl.conf

 

name = 'centos 7'
kernel = '/usr/lib/xen/boot/pv-grub-x86_64.gz'
extra  = '(hd0,0)/boot/grub/menu.lst'

... <rest of the domU configuration file> ...

Ways forward

  • wait for pv-grub2 support in the dom0 hypervisor (that can read '/boot/grub2/grub.cfg')
  • automatically generate a legacy v1 grub '/boot/grub/menu.lst'

Links

Appendices

Summary from srkykzm

The article 'Linnode migrate to pv grub' can be distilled down to:
# yum -y install yum-plugin-post-transaction-actions
# cat > /etc/yum/post-actions/grub1-kernel.action <<EOF
kernel*:remove:grubby --grub -o /boot/grub/menu.lst --remove-kernel=/boot/vmlinuz-\$ver-\$rel.\$arch
kernel*:install:grubby --grub -o /boot/grub/menu.lst --add-kernel /boot/vmlinuz-\$ver-\$rel.\$arch --args "root=LABEL=root" --title "Centos 7 (\$ver-\$rel.\$arch)" --initrd=/boot/initramfs-\$ver-\$rel.\$arch.img --copy-default --make-default
EOF

Note: 

  1. The 'yum-plugin-post-transaction-actions' package is part of the current base CentOS v7 distro.
  2. The root filesystem is matched by the label 'root' - not the device name

pv-grub prompt

If the boot fails the pv-grub prompt is shown with the version number (it is not grub2)

    GNU GRUB  version 0.97  (1048576K lower / 0K upper memory)

       [ Minimal BASH-like line editing is supported.   For
         the   first   word,  TAB  lists  possible  command
         completions.  Anywhere else TAB lists the possible
         completions of a device/filename. ]

grubdom>

Manual verfication of grub menu.lst

From the grub prompt pressing tab will provide auto-completion. This can be useful to verify the root path is correct:

grubdom> root (hd
 Possible disks are:  hd0 hd1 hd2

grubdom> root (hd0,0)/boot/grub
 Possible files are: grub2 grub

grubdom> root (hd0,0)/boot/grub/
 Possible files are: splash.xpm.gz menu.lst

grubdom> root (hd0,0)/boot/grub/menu.lst
 Filesystem type is ext2fs, partition type 0x83

Boot log

# xl create -c khaki
Parsing config from khaki
Xen Minimal OS!
  start_info: 0xd31000(VA)
    nr_pages: 0x70800
  shared_inf: 0x52933000(MA)
     pt_base: 0xd34000(VA)
nr_pt_frames: 0xb
    mfn_list: 0x9ad000(VA)
   mod_start: 0x0(VA)
     mod_len: 0
       flags: 0x0
    cmd_line: (hd0,0)/boot/grub/menu.lst
  stack:      0x96c2c0-0x98c2c0
MM: Init
      _text: 0x0(VA)
     _etext: 0x7dba3(VA)
   _erodata: 0x99000(VA)
     _edata: 0x9eea0(VA)
stack start: 0x96c2c0(VA)
       _end: 0x9ac8e0(VA)
  start_pfn: d42
    max_pfn: 70800
Mapping memory range 0x1000000 - 0x70800000
setting 0x0-0x99000 readonly
skipped 0x1000
MM: Initialise page allocator for 10bf000(10bf000)-70800000(70800000)
MM: done
Demand map pfns at 70801000-2070801000.
Heap resides at 2070802000-4070802000.
Initialising timer interface
Initialising console ... done.
gnttab_table mapped at 0x70801000.
Initialising scheduler
Thread "Idle": pointer: 0x2070802050, stack: 0x1460000
Thread "xenstore": pointer: 0x2070802800, stack: 0x1470000
xenbus initialised on irq 1 mfn 0x861ebe
Thread "shutdown": pointer: 0x2070802fb0, stack: 0x1480000
Dummy main: start_info=0x98c3c0
Thread "main": pointer: 0x2070803760, stack: 0x1490000
"main" "(hd0,0)/boot/grub/menu.lst"
vbd 51712 is hd0
******************* BLKFRONT for device/vbd/51712 **********


Shutting down ()
Shutdown requested: 3
Thread "shutdown" exited.
backend at /local/domain/0/backend/vbd/54/51712
16777216 sectors of 512 bytes
**************************
vbd 51728 is hd1
******************* BLKFRONT for device/vbd/51728 **********


backend at /local/domain/0/backend/vbd/54/51728
4194304 sectors of 512 bytes
**************************
vbd 51744 is hd2
******************* BLKFRONT for device/vbd/51744 **********


backend at /local/domain/0/backend/vbd/54/51744
67108864 sectors of 512 bytes
**************************

    GNU GRUB  version 0.97  (1843200K lower / 0K upper memory)

 +-------------------------------------------------------------------------+
 | CentOS 7 vmlinuz-3.10.0-123.13.2.el7.x86_64                             |
 |                                                                         |
 |                                                                         |
 |                                                                         |
 |                                                                         |
 |                                                                         |
 |                                                                         |
 |                                                                         |
 |                                                                         |
 |                                                                         |
 |                                                                         |
 |                                                                         |
 +-------------------------------------------------------------------------+
    Use the ^ and v keys to select which entry is highlighted.
    Press enter to boot the selected OS, 'e' to edit the
    commands before booting, or 'c' for a command-line.


    The highlighted entry will be booted automatically in 1 seconds.
  Booting 'CentOS 7 vmlinuz-3.10.0-123.13.2.el7.x86_64'

root (hd0,0)
 Filesystem type is ext2fs, partition type 0x83
kernel /boot/vmlinuz-3.10.0-123.13.2.el7.x86_64 root=LABEL=/ ro vconsole.keymap
=us console=hvc0 selinux=0 crashkernel=auto vconsole.font=latarcyrheb-sun16
initrd /boot/initramfs-3.10.0-123.13.2.el7.x86_64.img

============= Init TPM Front ================
Tpmfront:Error Unable to read device/vtpm/0/backend-id during tpmfront initialization! error = ENOENT
Tpmfront:Info Shutting down tpmfront
close blk: backend=/local/domain/0/backend/vbd/54/51712 node=device/vbd/51712
close blk: backend=/local/domain/0/backend/vbd/54/51728 node=device/vbd/51728
close blk: backend=/local/domain/0/backend/vbd/54/51744 node=device/vbd/51744
[    0.000000] Initializing cgroup subsys cpuset
[    0.000000] Initializing cgroup subsys cpu
[    0.000000] Initializing cgroup subsys cpuacct
[    0.000000] Linux version 3.10.0-123.13.2.el7.x86_64 (builder@kbuilder.dev.centos.org) (gcc version 4.8.2 20140120 
     (Red Hat 4.8.2-16) (GCC) ) #1 SMP Thu Dec 18 14:09:13 UTC 2014
[    0.000000] Command line: root=LABEL=/ ro vconsole.keymap=us console=hvc0 selinux=0 crashkernel=auto vconsole.font=latarcyrheb-sun16
[    0.000000] ACPI in unprivileged domain disabled
[    0.000000] e820: BIOS-provided physical RAM map:
[    0.000000] Xen: [mem 0x0000000000000000-0x000000000009ffff] usable
[    0.000000] Xen: [mem 0x00000000000a0000-0x00000000000fffff] reserved
[    0.000000] Xen: [mem 0x0000000000100000-0x00000001ffffffff] usable
Document Actions