In this post, we are about to know how to resolve I/O error while listing the logical volumes or volume groups in Linux. After removing anyone of SAN disk from any Volume Group it will print the I/O error whenever we try to run PVS, VGS or LVS.
This can be only in case when someone not following the proper procedures to remove the SAN LUN’S from a Linux server.
Have a read to know about How to remove a SAN disk or Storage device (LUN) from Linux server.
root@PRODUORACLEDB03 ~]# lvs
/dev/Volgrp_Archive_backup/LogVol_Archive: read failed after 0 of 4096 at 219901: Input/output error
/dev/Volgrp_Archive_backup/LogVol_Archive: read failed after 0 of 4096 at 219902: Input/output error
/dev/Volgrp_Archive_backup/LogVol_Archive: read failed after 0 of 4096 at 0: Input/output error
/dev/Volgrp_Archive_backup/LogVol_Archive: read failed after 0 of 4096 at 4096: Input/output error
/dev/sdcy: read failed after 0 of 4096 at 2199014858752: Input/output error
Method 1: Removing Stale Disks
Above error can be resolved by removing the stale disks. It’s not required to unmount the file system in this method, But we need to be very careful while performing below steps.
List out VG information using vgdisplay
# vgdisplay Volgrp_Archive_backup
Remove all the stale devices by running dmsetup and by running echo by appending 1 into each device paths, or devices.
# dmsetup remove /dev/Volgrp_Archive_backup/*
# echo 1 > /sys/block/sdcy/device/delete
Method 2: Removing Stale Disk using vgchange
Using this method, we need to unmount the file system.
To find the exact VG which have removed disk’s do a scan.
# umount /Archive_backup
Before bringing down the volume group offline, we need to bring down the Logical volume offline.
# lvchange -an LogVol_Archive
Once above logical volume offline, Make the volume group offline.
# vgchange -an Volgrp_Archive_backup
Now run “vgscan” to Search for all volume groups.
Now its time to bring back the VG and LV online in reverse order. First, make the volume Group online.
# vgchange -ay Volgrp_Archive_backup
By following make the Logical volume online to mount the file system.
# lvchange -ay LogVol_Archive
# mount /dev/mapper/Volgrp_Archive_backup-LogVol_Archive /Archive_backup
Using any one of the methods we can recover from stale disk issues in Linux.
In case if someone not followed the right procedure to remove a SAN disk we may notice I/O error while trying to list the PV, VG and LV. This can be cleared using above two steps. However, we need to avoid such practice while removing SAN LUN’s to be free from error. Let catch you in next troubleshooting guide, till then subscribe to our newsletter and feed to keep you update.