CentOS 7 UEFI Install on appliance with mirrored GPT partitions
[GPT, UEFI, Mirror, Raid1]
The following notes are on installing CentOS v7 on a physical appliance with two small SSD's. Every practical step is taken overcome a single disk failure.
Key aspects:
- The BIOS is configured for UEFI boot
- The installation is done with UEFI boot from a USB flash drive based image (dd the iso onto the drive)
- The disks are partitioned with a GUID partition table (GPT)
- Linux software RAID 1 (mirror) is used to mirror the partitions (not whole device)
- A non-RAIDed UEFI FAT based partition for booting
- the 'second' disk has a copy of the files
- Anaconda is limited in it's support of creating identical disks
- Simple filesystems are used
- no xfs, just ext4
- no LVM
- No hardware RAID
Template Disk
Install CentOS 7 using anaconda. Manually create the following paritions during the installation:
- a 100M EFI System Partition (ESP) FAT16 partition
- a 1G '/boot' ext4 RAID1 partition
- a 4G swap RAID1 partition
- 1 12G '/' (root) ext4 RAID1 partition
Ananonda can do everything except helping create a copy/mirror of the EFI partition as parition number one on the secondary disk. Let anaconda partition and work with the second disk, but the next step discards that for the second disk after the installation is complete..
# parted /dev/sda GNU Parted 3.1 Using /dev/sda Welcome to GNU Parted! Type 'help' to view a list of commands. (parted) unit s (parted) print Model: ATA KINGSTON SMS200S (scsi) Disk /dev/sda: 58626288s Sector size (logical/physical): 512B/512B Partition Table: gpt Disk Flags: Number Start End Size File system Name Flags 1 2048s 206847s 204800s fat16 EFI System Partition boot 2 206848s 8402943s 8196096s raid 3 8402944s 10450943s 2048000s ext4 raid 4 10450944s 35043327s 24592384s raid
Remove /dev/sdb from the RAID
The second disk is part of the three RAID mirrors
# cat /proc/mdstat Personalities : [raid1] md125 : active raid1 sda4[0] sdb4[1] 12287872 blocks super 1.2 [2/2] [UU] bitmap: 0/1 pages [0KB], 65536KB chunk md126 : active raid1 sda3[0] sdb2[1] 1023936 blocks super 1.0 [2/2] [UU] md127 : active raid1 sda2[0] sdb1[1] 4095936 blocks super 1.2 [2/2] [UU] unused devices: <none>
Fail and remote the three partitions from the mirrors:
# mdadm --fail /dev/md125 sdb4 # mdadm --remove /dev/md125 sdb4 # mdadm --fail /dev/md126 sdb2 # mdadm --remove /dev/md126 sdb2 # mdadm --fail /dev/md127 sdb1 # mdadm --remove /dev/md127 sdb1
Unmount the secondary fat16 filesystem created during the installation:
# umount /boot/efi2
# cat /proc/mdstat Personalities : [raid1] md125 : active raid1 sda4[0] 12287872 blocks super 1.2 [2/1] [U_] bitmap: 1/1 pages [4KB], 65536KB chunk md126 : active raid1 sda3[0] 1023936 blocks super 1.0 [2/1] [U_] md127 : active raid1 sda2[0] 4095936 blocks super 1.2 [2/1] [U_] unused devices: <none>
Repartition the second disk
Partition
Use 'parted' to recreate the second device using the dimensions (units of sectors) from the frist device:
# parted /dev/sdb mktable gpt unit s mkpart primary fat16 2048s 206847s mkpart primary ext4 206848s 8402943s mkpart primary linux-swap 8402944s 10450943s mkpart primary ext4 10450944s 35043327s name 1 "EFI System Partition" name 2 /boot name 3 swap name 4 / set 1 boot on set 2 raid on set 3 raid on set 4 raid on quit
FAT EFI
Create a FAT16 filesystem on the backup EFI boot partition (Note: If the system was being used with Windows then a FAT32 filesystem would be required):
# mkfs.vfat -F 16 -n 'EFI System Partition' /dev/sdb1
Update the '/etc/fstab' with the UUID of the of new FAT filesystem (use 'lsblk -f' to find the id) and then remount the secondary EFI filesystem (which was mounted as '/boot/efi2' during the anaconda install):
# mount -a
Manually copy the EFI files from the first disk to the second:
# cp -av /boot/efi/. /boot/efi2/
Mirrors
Remirror the three partitions to 'boot', swap and '/':
mdadm --add /dev/md125 /dev/sdb4 mdadm --add /dev/md126 /dev/sdb3 mdadm --add /dev/md127 /dev/sdb2
EFI Boot
Ensure both disks are added to the UEFI NVRAM. Using a label that includes the disk serial number (if part of it) is useful:
# efibootmgr -c -d /dev/sdb -g -p 1 -L "CentOS 46034436" -l '\EFI\centos\shim.efi' # efibootmgr -c -d /dev/sda -g -p 1 -L "CentOS 4603410E" -l '\EFI\centos\shim.efi'
Filesystems
When the system boots in a degraded mode the /boot/efi or /boot/efi2 ESP filesystems may not be available. So that systemd doesn't halt the boot process add a 'nofail' option to the '/etc/fstab' for the two ESP filesystems:
UUID=34d2ee88-3ed3-4aa6-9279-f9733bca95d5 / ext4 defaults 1 1 UUID=346ba696-2d7e-4f4d-b56d-4d5d0cf4da21 /boot ext4 defaults 1 2 UUID=F6CE-F5D0 /boot/efi vfat umask=0077,shortname=winnt,nofail 0 0 UUID=A6B3-2CF2 /boot/efi2 vfat umask=0077,shortname=winnt,nofail 0 0 UUID=1e4da845-94a2-4672-9505-5cbe5e8e716f swap swap defaults 0 0
Links
- https://www.gnu.org/software/parted/manual/html_chapter/parted_2.html
- http://en.wikipedia.org/wiki/Unified_Extensible_Firmware_Interface
- http://unix.stackexchange.com/questions/69214/partitioning-efi-machine-with-two-ssd-disks-in-mirror
- http://www.freedesktop.org/software/systemd/man/systemd.mount.html
- http://www.thomas-krenn.com/en/wiki/Restore_Ubuntu_UEFI_Boot_Entries_after_BIOS_Update
Appendices
lsblk
# lsblk -f NAME FSTYPE LABEL UUID MOUNTPOINT sda ├─sda1 vfat F6CE-F5D0 /boot/efi ├─sda2 linux_raid_member localhost:swap 32bf0d04-5da1-f473-62f5-1c568b489995 │ └─md127 swap swap 1e4da845-94a2-4672-9505-5cbe5e8e716f [SWAP] ├─sda3 linux_raid_member localhost:boot 9826d78d-61eb-8315-818f-196cd5e074a5 │ └─md126 ext4 /boot 346ba696-2d7e-4f4d-b56d-4d5d0cf4da21 /boot └─sda4 linux_raid_member localhost:root 6f2768ed-9df6-643a-0d82-3c5966bfc606 └─md125 ext4 34d2ee88-3ed3-4aa6-9279-f9733bca95d5 / sdb ├─sdb1 vfat EFI SYSTEM A6B3-2CF2 /boot/efi2 ├─sdb2 linux_raid_member localhost:swap 32bf0d04-5da1-f473-62f5-1c568b489995 │ └─md127 swap swap 1e4da845-94a2-4672-9505-5cbe5e8e716f [SWAP] ├─sdb3 linux_raid_member localhost:boot 9826d78d-61eb-8315-818f-196cd5e074a5 │ └─md126 ext4 /boot 346ba696-2d7e-4f4d-b56d-4d5d0cf4da21 /boot └─sdb4 linux_raid_member localhost:root 6f2768ed-9df6-643a-0d82-3c5966bfc606 └─md125 ext4 34d2ee88-3ed3-4aa6-9279-f9733bca95d5 /
EFI Files
# cp -av /boot/efi/. /boot/efi2/ ‘/boot/efi/./EFI/centos’ -> ‘/boot/efi2/./EFI/centos’ ‘/boot/efi/./EFI/centos/fonts’ -> ‘/boot/efi2/./EFI/centos/fonts’ ‘/boot/efi/./EFI/centos/fonts/unicode.pf2’ -> ‘/boot/efi2/./EFI/centos/fonts/unicode.pf2’ ‘/boot/efi/./EFI/centos/grub.cfg’ -> ‘/boot/efi2/./EFI/centos/grub.cfg’ ‘/boot/efi/./EFI/centos/gcdx64.efi’ -> ‘/boot/efi2/./EFI/centos/gcdx64.efi’ ‘/boot/efi/./EFI/centos/grubx64.efi’ -> ‘/boot/efi2/./EFI/centos/grubx64.efi’ ‘/boot/efi/./EFI/centos/BOOT.CSV’ -> ‘/boot/efi2/./EFI/centos/BOOT.CSV’ ‘/boot/efi/./EFI/centos/MokManager.efi’ -> ‘/boot/efi2/./EFI/centos/MokManager.efi’ ‘/boot/efi/./EFI/centos/shim-redhat.efi’ -> ‘/boot/efi2/./EFI/centos/shim-redhat.efi’ ‘/boot/efi/./EFI/centos/shim.efi’ -> ‘/boot/efi2/./EFI/centos/shim.efi’ ‘/boot/efi/./EFI/BOOT’ -> ‘/boot/efi2/./EFI/BOOT’ ‘/boot/efi/./EFI/BOOT/BOOTX64.EFI’ -> ‘/boot/efi2/./EFI/BOOT/BOOTX64.EFI’ ‘/boot/efi/./EFI/BOOT/fallback.efi’ -> ‘/boot/efi2/./EFI/BOOT/fallback.efi’
efibootmgr
# efibootmgr -c -d /dev/sdb -p 1 -L "CentOS" -l '\EFI\centos\shim.efi' ** Warning ** : Boot0000 has same label CentOS BootCurrent: 0000 Timeout: 1 seconds BootOrder: 0002,0000,0001 Boot0000* CentOS Boot0001* UEFI: KINGSTON SMS200S330G Boot0002* CentOS
gdisk
# gdisk -l /dev/sda GPT fdisk (gdisk) version 0.8.6 Partition table scan: MBR: protective BSD: not present APM: not present GPT: present Found valid GPT with protective MBR; using GPT. Disk /dev/sda: 58626288 sectors, 28.0 GiB Logical sector size: 512 bytes Disk identifier (GUID): 63EE03DD-4F8F-49C5-9564-56179029BD32 Partition table holds up to 128 entries First usable sector is 34, last usable sector is 58626254 Partitions will be aligned on 2048-sector boundaries Total free space is 23584941 sectors (11.2 GiB) Number Start (sector) End (sector) Size Code Name 1 2048 206847 100.0 MiB EF00 EFI System Partition 2 206848 8402943 3.9 GiB FD00 3 8402944 10450943 1000.0 MiB FD00 4 10450944 35043327 11.7 GiB FD00
# gdisk -l /dev/sdb GPT fdisk (gdisk) version 0.8.6 Partition table scan: MBR: protective BSD: not present APM: not present GPT: present Found valid GPT with protective MBR; using GPT. Disk /dev/sdb: 58626288 sectors, 28.0 GiB Logical sector size: 512 bytes Disk identifier (GUID): 92063063-2A9C-4D91-888A-3FDD49CEA0D7 Partition table holds up to 128 entries First usable sector is 34, last usable sector is 58626254 Partitions will be aligned on 2048-sector boundaries Total free space is 23584941 sectors (11.2 GiB) Number Start (sector) End (sector) Size Code Name 1 2048 206847 100.0 MiB EF00 EFI System Partition 2 206848 8402943 3.9 GiB FD00 /boot 3 8402944 10450943 1000.0 MiB FD00 swap 4 10450944 35043327 11.7 GiB FD00 /