One of the most common problem in Redhat Linux is “read only root filesystem” when there is issue with hardware or SAN sub-systems .To fix the “read only filesystem” issue ,you need to reboot the server.But while rebooting the server, you will definitely encounter many issues and that requires a manual intervention by logging in with the root. Unfortunately you may not logging as root if root filesystem is corrupted and it will not accept the root password. In this case, you need to boot the system in rescue mode using Redhat Linux DVD.
Here is the steps which you need to try before rebooting the system in rescue mode using Redhat Linux DVD.
1.Reboot the system and see whether system is automatically booting in normal mode with root filesystem in read/write.While booting automatically, it will check the root filesystem filesystem integrity using fsck.
2.Some times system will go in to maintenance mode after after rebooting.In this case, system will prompt for root password. Try to Login and run fsck on root filesystem to fix the issue.
3.If there is any problem with /etc/rc.sysinit(/etc/rc.d/rc.sysinit) , system will always boot and make the root filesystem in read-only mode.In this case, you need restore the file from backup or copy the file from other similar redhat server.
4.If the system is booted in maintenance mode and not accpeting the root password, then you need to boot the system in rescue mode using redhat Linux DVD.
Rescue mode system recovery:
1.Insert the redhat DVD and boot the system in rescue mode.
2.Select the language as “English” and continue.
3.Select “No” to skip the networking.
4.Select “skip” to run fsck on root filesystem.
5.If you are not using “LVM” for root filesystem,execute fsck on the device which contains the root filesystem
UnixArena# e2fsck -f /dev/sda1
e2fsck 1.51.12 (17-May-2013)
[...]
6.If you are using LVM for root filesystem, You need to activate the logical volumes using below method.
# lvm pvscan
PV /dev/sda1 VG VolGroup00 lvm2 [996.00 MB / 0 free]
Total: 1 [8.48 GB] / in use: 1 [8.48 GB] / in no VG: 0 [0 ]
# lvm vgscan
Reading all physical volumes. This may take a while...
Found volume group "VolGroup00" using metadata type lvm2
# lvm lvscan
INACTIVE '/dev/VolGroup00/LogVol00' [8.48 GB] inherit
# lvm vgchange -ay
1 logical volume(s) in volume group "VolGroup00" now active
7.Run fsck on the root filesystem logical volume to check the integrity.
# e2fsck -f /dev/mapper/VolGroup00/LogVol00
e2fsck 1.51.12 (17-May-2013)
[...]
8.Exit the rescue mode and boot the system normally.
Hope this will fix the issue.
I have seen many times , redhat Linux which is running on VMWARE ESX may face this issue.In these cases , you need to follow the below steps.
1. Update VMware tools .
2. Migrate the affected virtual machine to other datastore and monitors the virtual machine. If issue doesn’t reappears, It could be problem with the storage. Engage the storage vendor.
2. Even after the storage migration, if issue reoccurs .Update the Linux kernel to the latest version.
3. Increase the SCSI timeout of each disk presented from VMWare as per the Redhat Linux article.
To troubleshoot further on VMWARE ESX for this issue, Please follow this article.
Thanks to www.vmwarearena.com
Thank you for visiting UnixArena. Please leave a comment if you have any doubt.