Introduction

There are times when you have a system that has a single boot partition that houses all of the important Linux files, such as /boot and /home, on a single root (/) filesystem. This is particularly true with cloud-based images and virtual machine images, such as those in QCOW2 format.

For example, here is the output of a freshly-built virtual machine from a CentOS7 QCOW2 cloud image:

# df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/vda1       8.0G  865M  7.2G  11% /
devtmpfs        441M     0  441M   0% /dev
tmpfs           463M     0  463M   0% /dev/shm
tmpfs           463M   13M  451M   3% /run
tmpfs           463M     0  463M   0% /sys/fs/cgroup
tmpfs            93M     0   93M   0% /run/user/1000
tmpfs            93M     0   93M   0% /run/user/0

You can see that the root (/) filesystem and all of the user files are stored in a single 8GB partition on /dev/vda1. In many cases, this situation would be fine. But what happens when you want to increase the capacity of root (/), or want to have separate partitions for /boot and /home?

In this article, we will explore a method for converting from a single partition that contains all of the files for Linux and user files into a system that has a separate partition for /boot and an LVM2 volume group containing the Linux root (/) filesystem.

The following is an example only. You can modify the LVM2 volumes and filesystems however you want, such as creating a separate volume for /var or /tmp.




Prepare for Migration

Step 1: Install required software

Most cloud images will not include the software we need to create volume groups and migrate data. Install LVM2 and rsync for this procedure:

# yum -y install lvm2 rsync

Step 2: Create the Volume Group and Logical Volume(s)

In our example, we will be migrating to a new disk. The current boot/root device is /dev/vda1 and we will be moving to a boot partition on /dev/vdb1 and a Volume Group on /dev/vdb2.

# lsblk
NAME   MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT 
vda    253:0    0    8G  0 disk 
└─vda1 253:1    0    8G  0 part /
vdb    253:16   0   25G  0 disk 

Partition the device with a 1GB partition for partition 1 and the remaining space as partition 2. Partition 1 should have the “bootable” flag set, and partition 2 should be type 8e, or “Linux LVM”.

For further information about partitioning disks with FDisk, please see this article.

# fdisk /dev/vdb
Welcome to fdisk (util-linux 2.23.2).

Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Device does not contain a recognized partition table
Building a new DOS disklabel with disk identifier 0xd921efb5.

Command (m for help): n
Partition type:
   p   primary (0 primary, 0 extended, 4 free)
   e   extended
Select (default p): 
Using default response p
Partition number (1-4, default 1): 
First sector (2048-52428799, default 2048): 
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-52428799, default 52428799): +1G
Partition 1 of type Linux and of size 1 GiB is set

Command (m for help): n
Partition type:
   p   primary (1 primary, 0 extended, 3 free)
   e   extended
Select (default p): 
Using default response p
Partition number (2-4, default 2): 
First sector (2099200-52428799, default 2099200): 
Using default value 2099200
Last sector, +sectors or +size{K,M,G} (2099200-52428799, default 52428799): 
Using default value 52428799
Partition 2 of type Linux and of size 24 GiB is set

Command (m for help): t
Partition number (1,2, default 2):  
Hex code (type L to list all codes): 8e
Changed type of partition 'Linux' to 'Linux LVM'

Command (m for help): a
Partition number (1,2, default 2): 1

Command (m for help): p

Disk /dev/vdb: 26.8 GB, 26843545600 bytes, 52428800 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0xd921efb5

   Device Boot      Start         End      Blocks   Id  System
/dev/vdb1   *        2048     2099199     1048576   83  Linux
/dev/vdb2         2099200    52428799    25164800   8e  Linux LVM

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

Now create the Physical Volume and Volume group.

For further information about LVM2 and creating Volume Groups and Logical Volumes, please see this article.

# pvcreate /dev/vdb2
  Physical volume "/dev/vdb2" successfully created.
# vgcreate rootvg /dev/vdb2
  Volume group "rootvg" successfully created

Create the Logical Volumes. We will be creating a 1 GB volume for swap and a volume for root (/) with the remaining space in the Volume Group. If you want to create additional volumes, this is the time to do it, but you will need to modify the subsequent steps accordingly.

# lvcreate -n swap -L 1G rootvg
  Logical volume "swap" created.
# lvcreate -n root -l 100%FREE rootvg
  Logical volume "root" created.

Format the filesystems. We will use EXT4 for /boot and XFS for root (/), but you may use other filesystem types as you see fit.

For further information about creating and mounting filesystems, please see this article.

# mkfs.ext4 /dev/vdb1
mke2fs 1.42.9 (28-Dec-2013)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
65536 inodes, 262144 blocks
13107 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=268435456
8 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks: 
	32768, 98304, 163840, 229376

Allocating group tables: done                            
Writing inode tables: done                            
Creating journal (8192 blocks): done
Writing superblocks and filesystem accounting information: done

# mkfs.xfs /dev/rootvg/root 
meta-data=/dev/rootvg/root       isize=512    agcount=4, agsize=1507072 blks
         =                       sectsz=512   attr=2, projid32bit=1
         =                       crc=1        finobt=0, sparse=0
data     =                       bsize=4096   blocks=6028288, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0 ftype=1
log      =internal log           bsize=4096   blocks=2943, version=2
         =                       sectsz=512   sunit=0 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0

# mkswap /dev/rootvg/swap 
Setting up swapspace version 1, size = 1048572 KiB
no label, UUID=f76bcb12-8100-432f-9c50-49306a65ffce

Perform the Migration

Step 3: Create directories to mount the volumes

In order to do the copy, we need to mount our new volumes into their target structure. Note that we do not need to activate swap at this time.

# mkdir /newroot
# mount /dev/rootvg/root /newroot/
# mkdir /newroot/boot
# mount /dev/vdb1 /newroot/boot/


We will also need to create a bind mount for the existing root (/) filesystem. This allows us to copy data without copying files we don’t need, like the tmpfs filesystems and /dev.

# mkdir /oldroot
# mount -o bind / /oldroot

Step 4: Copy the data

We will use rsync to copy the data, but other tools should work just as well, so long as they preserve ownership and permissions. This step could take some time depending on how much data there is to copy.

# rsync -avx /oldroot/ /newroot/

...

sent 890,446,727 bytes  received 455,945 bytes  23,757,404.59 bytes/sec
total size is 888,695,623  speedup is 1.00

Step 5: Update configuration files for the new root disk

There are several configuration files that need to be updated in order for the system to boot properly. It is very important that these files are modified carefully, or else the system may become unbootable!

Let’s start with the new fstab. Modify the file /newroot/etc/fstab and make modifications for the new devices that we created.

First, we need to know the UUID of the new /boot filesystem on /dev/vdb1. (Hint: It’s the one that ends in “a13a”!)

# blkid
/dev/vda1: UUID="de86ba8a-914b-4104-9fd8-f9de800452ea" TYPE="xfs" 
/dev/vdb1: UUID="3dfa0ae6-e70b-4600-a740-007a0d72a13a" TYPE="ext4" 
/dev/vdb2: UUID="xsttrD-aJ6R-Sm2k-OEbq-k7wF-eJfG-k9mp7h" TYPE="LVM2_member" 
/dev/mapper/rootvg-swap: UUID="f76bcb12-8100-432f-9c50-49306a65ffce" TYPE="swap" 
/dev/mapper/rootvg-root: UUID="49f23775-802b-4f2e-8b82-cecb81118a9d" TYPE="xfs" 

Here is the old fstab:

# cat /newroot/etc/fstab 
UUID=de86ba8a-914b-4104-9fd8-f9de800452ea /         xfs     defaults        0 0

And here is the new one:

# cat /newroot/etc/fstab 
/dev/mapper/rootvg-root                   /         xfs     defaults        0 0
UUID=3dfa0ae6-e70b-4600-a740-007a0d72a13a /boot     ext4    defaults        1 2
/dev/mapper/rootvg-swap                   swap      swap    defaults        0 0

Now we need to update the Grub defaults file in /newroot/etc/default/grub.

Here is the line we need to edit in the old file:

# grep CMDLINE /newroot/etc/default/grub 
GRUB_CMDLINE_LINUX="console=tty0 crashkernel=auto console=ttyS0,115200"

And here is the new line:

# grep CMDLINE /newroot/etc/default/grub 
GRUB_CMDLINE_LINUX="crashkernel=auto console=ttyS0,115200 console=tty0 rd.lvm.lv=rootvg/root"

VERY IMPORTANT NOTE!
Notice there are multiple differences in the two versions. The most important for booting is the addition of “rd.lvm.lv=rootvg/root”. However, we also moved the text “console=tty0” from the beginning of the line to after the text “console=ttyS0,115200”. This is very important for SELinux auto labelling that we will do later. Failure to do this correctly will lead to a system that will boot, but login will fail (/bin/bash will fail to execute).

Step 6: Modify boot parameters

Next, we need to create a new grub.cfg file, a new initramfs, and install grub to the MBR of the new disk. To do these things, we need to create a chroot jail on our new root volume.

The commands we will run require access to /dev, /sys, and /proc, but those don’t exist in a chroot jail, so we have to bind mount them under the mountpoint of the new root volume (/newroot).

# mount -o bind /dev /newroot/dev
# mount -o bind /sys /newroot/sys
# mount -o bind /proc /newroot/proc

Now we go into the chroot jail. You can tell we are there because df will show us our new volumes. Note: it is normal that /proc and /sys do not appear in the df output.

# chroot /newroot/
# df -h
Filesystem               Size  Used Avail Use% Mounted on
/dev/mapper/rootvg-root   23G  840M   23G   4% /
devtmpfs                 441M     0  441M   0% /dev
/dev/vdb1                976M  109M  801M  12% /boot

Create a new grub.cfg file. The errors about lvmetad can be ignored.

# grub2-mkconfig -o /boot/grub2/grub.cfg 
Generating grub configuration file ...
  WARNING: Failed to connect to lvmetad. Falling back to device scanning.
  WARNING: Failed to connect to lvmetad. Falling back to device scanning.
Found linux image: /boot/vmlinuz-3.10.0-862.14.4.el7.x86_64
Found initrd image: /boot/initramfs-3.10.0-862.14.4.el7.x86_64.img
Found linux image: /boot/vmlinuz-0-rescue-5003025f93c1a84914ea5ae66519c100
Found initrd image: /boot/initramfs-0-rescue-5003025f93c1a84914ea5ae66519c100.img
  WARNING: Failed to connect to lvmetad. Falling back to device scanning.
  WARNING: Failed to connect to lvmetad. Falling back to device scanning.
done

Create a new initramfs file that contains the modules for LVM. It’s important that you use the correct kernel version for this. In this example, there is only one version installed, so it’s easy.

# dracut -f -v /boot/initramfs-3.10.0-862.14.4.el7.x86_64.img 
Executing: /usr/sbin/dracut -f -v /boot/initramfs-3.10.0-862.14.4.el7.x86_64.img

...

*** Creating image file done ***
*** Creating initramfs image file '/boot/initramfs-3.10.0-862.14.4.el7.x86_64.img' done ***

Now we need to install Grub to the boot sector of /dev/vdb. We need the –recheck option so that grub will write a correct device map.

# grub2-install --recheck /dev/vdb
Installing for i386-pc platform.
Installation finished. No error reported.

Finally, we need to ensure that SELinux relabels the copied files when the system boots. There are many ways to do this, but the easiest is just to create the file /.autorelabel. If your system is not using SELinux, this step can be skipped. If you’re not sure, just do it; it won’t hurt anything.

# touch /.autorelabel

Exit the chroot jail. And shutdown (not reboot!) the server.

# exit
# shutdown -h now

Step 7: Modify the boot parameters of the VM or physical server.

If the server is a virtual machine, then remove the original boot disk. It is probably best to simply detach it instead of deleting it, just in case there are problems and you need to boot off of it again later.

If the server is physical, then you will need to change the boot order in the BIOS to boot off of the new disk first.

(In the demonstration environment, the virtual machine runs on oVirt. oVirt remembers the order of disks when they are added to the system, so removing the original boot disk puts the DVD-ROM drive first in the boot order. When the system POSTs and finds no DVD in the drive, it fails to boot. To fix this, the new disk needs to have the “bootable” option set on the virtual disk, which moves the new disk to the first in the boot order).



Step 8: Boot the server

If you’ve done things properly, you should be presented with the Grub boot menu. If you wish, you can edit the boot menu option to validate the line starting with “linux16”, which is the kernel command line. It should look like the one below with your root Volume Group and Volume at the end of the line, and your /boot UUID in the “search” line.

...

	search --no-floppy --fs-uuid --set=root 3dfa0ae6-e70b-4600-a740-007a0d72a13a

	linux16 /vmlinuz-0-rescue-5003025f93c1a84914ea5ae66519c100 root=/dev/mapper/rootvg-root ro crashkernel=auto console=ttyS0,115200 console=tty0 rd.lvm.lv=rootvg/root 

...

Once you are satisfied that the boot menu option is correct, boot the system.

If you are watching the console of your server, you will see the boot process pause with messages similar to the following:

...

*** Warning -- SELinux targeted policy relabel is required.
*** Relabeling could take a very long time, depending on file
*** system size and speed of hard drives.

...

When the SELinux relabeling process has completed, the system will reboot again automatically.

When you login after the final reboot, you will see that the system has been migrated to LVM with a separate /boot partition and swap already activated.

# df -h
Filesystem               Size  Used Avail Use% Mounted on
/dev/mapper/rootvg-root   23G  839M   23G   4% /
devtmpfs                 444M     0  444M   0% /dev
tmpfs                    463M     0  463M   0% /dev/shm
tmpfs                    463M   13M  451M   3% /run
tmpfs                    463M     0  463M   0% /sys/fs/cgroup
/dev/vda1                976M  103M  807M  12% /boot
tmpfs                     93M     0   93M   0% /run/user/0

# swapon -s
Filename			Type		Size	Used	Priority
/dev/dm-1                    	partition	1048572	0	-1

Conclusion

Performing maintenance on a root (/) filesystem is always a hassle, and so having a system running from LVM2 can add a great deal of flexibility for disk maintenance, such as increasing a filling filesystem. As we have shown above, migrating from a single partition to LVM2 is not so difficult and does not require a long outage if it’s done right.

Pre-installed virtual machine images are very beneficial for starting up a system quickly, but they are not always built for use over a long period of time. This procedure will help to prevent you from being stuck with a system that does not meet your needs and will allow you to expand your storage as your demand grows.

LEAVE A REPLY

Please enter your comment!
Please enter your name here