2023-11-13 09:20 PM
Hi Team,
i have replaced Solid State Disk (SSD) of concentrator DAC with another capacity SSD.
previous capacity 186.310 GB of SSD. Now i have replaced it with 366.796 GB.
after replace SSD is showing status "US" [Unconfigured (Good) and Solid-State Drive] and warning "Physical disk problems have been found."
details are
[root@DC2-CON root] # ./nwraidutil.pl
RSA NetWitness/Security Analytics RAID Utility Version 2015.12.01
Using /opt/MegaRAID/MegaCli/MegaCli64
------------------------
Logical Disk Information
------------------------
Physical Drive State Legend
-------------------------------------------------------------------------------------
B Unconfigured(Bad) O Online
C Copyback R Rebuild
D Dedicated Hotspare and associated virtual drive number S Solid-State Drive
E Hotspare prefers same enclosure U Unconfigured(Good)
F Foreign X Offline
G Global hotspare ! Failed
I Hotspare is revertible ? Unknown state
M Missing
-------------------------------------------------------------------------------------
NOTE: 'E' does not prohibit a hotspare from being used in another enclosure, it is merely a preference
Logical Drive State Legend
------------------------------------------------
D Degraded X Offline
O Optimal ! Failed
P Partially Degraded ? Unknown state
R Rebuild
------------------------------------------------
Adapter: 0 - PERC H710P Mini
Virtual Disk: 0 (O) - Found 2 of 2 {Raid Level 1, 136.125 GB, 128 KB Stripe Size, WriteBack, ReadAdaptive, Cached, Write Cache OK if Bad BBU}
PD: 0 Enclosure: 32 Slot: 0 (O) 136.732 GB TOSHIBA MK1401GRRB DB0252A0A03UFQ87
PD: 1 Enclosure: 32 Slot: 1 (O) 136.732 GB TOSHIBA MK1401GRRB DB0252B0A034FQ87
Virtual Disk: 1 (O) - Found 2 of 2 {Raid Level 1, 931.0 GB, 128 KB Stripe Size, WriteBack, ReadAdaptive, Cached, Write Cache OK if Bad BBU}
PD: 0 Enclosure: 32 Slot: 2 (O) 931.512 GB SEAGATE ST91000640SS AS069XG2TQ6K
PD: 1 Enclosure: 32 Slot: 3 (O) 931.512 GB SEAGATE ST91000640SS AS069XG2SDVB
Adapter: 1 - PERC H810 Adapter
Virtual Disk: 0 (O) - Found 7 of 7 {Raid Level 5, 10.913 TB, 128 KB Stripe Size, WriteBack, ReadAdaptive, Cached, Write Cache OK if Bad BBU}
PD: 0 Enclosure: 63 Slot: 7 (O) 1.819 TB HITACHI HUS72302CLAR2000C1D6YGKT4W2K
PD: 1 Enclosure: 63 Slot: 8 (O) 1.819 TB HITACHI HUS72302CLAR2000C1D6YGKSVK4K
PD: 2 Enclosure: 63 Slot: 9 (O) 1.819 TB HITACHI HUS72302CLAR2000C1D6YGKT9SYK
PD: 3 Enclosure: 63 Slot: 10 (O) 1.819 TB HITACHI HUS72302CLAR2000C1D6YGKSXDYK
PD: 4 Enclosure: 63 Slot: 11 (O) 1.819 TB HITACHI HUS72302CLAR2000C1D6YGKSXS4K
PD: 5 Enclosure: 63 Slot: 12 (O) 1.819 TB HITACHI HUS72302CLAR2000C1D6YGKSTB7K
PD: 6 Enclosure: 63 Slot: 13 (O) 1.819 TB HITACHI HUS72302CLAR2000C1D6YGKSXBBK
Virtual Disk: 1 (D) - Found 6 of 7 {Raid Level 5, 1.087 TB, 128 KB Stripe Size, WriteBack, ReadAdaptive, Cached, Write Cache OK if Bad BBU}
PD: 0 Enclosure: 63 Slot: 0 (OS) 186.310 GB HITACHI HUSRL402 CLAR200C190XTVUNYRA
PD: 1 Enclosure: 63 Slot: 1 (OS) 186.310 GB HITACHI HUSRL402 CLAR200C190XTVTK14A
PD: 2 Enclosure: 63 Slot: 2 (OS) 186.310 GB HITACHI HUSRL402 CLAR200C190XTVUY4UA
PD: 3 Enclosure: 63 Slot: 3 (OS) 186.310 GB HITACHI HUSRL402 CLAR200C190XTVUY8RA
PD: 4 Enclosure: 63 Slot: 4 (OS) 186.310 GB HITACHI HUSRL402 CLAR200C190XTVTZR7A
PD: 6 Enclosure: 63 Slot: 6 (OS) 186.310 GB HITACHI HUSRL402 CLAR200C190XTVU1TWA
WARNING: Logical problems have been found.
WARNING: Drive Count Mismatch!
-------------------------
Physical Disk Information
-------------------------
Physical Drive State Legend
-------------------------------------------------------------------------------------
B Unconfigured(Bad) O Online
C Copyback R Rebuild
D Dedicated Hotspare and associated virtual drive number S Solid-State Drive
E Hotspare prefers same enclosure U Unconfigured(Good)
F Foreign X Offline
G Global hotspare ! Failed
I Hotspare is revertible ? Unknown state
M Missing
-------------------------------------------------------------------------------------
NOTE: 'E' does not prohibit a hotspare from being used in another enclosure, it is merely a preference
Adapters found: 2
Adapter 0 (PERC H710P Mini) enclosures found: 1
Adapter 0 (PERC H710P Mini) enclosure 32 slots found: 4
Encl Slot State P.Fail.Count Raw Size Drive FW Inquiry Data
32 0 (O) 0 136.732 GB DB02 TOSHIBA MK1401GRRB DB0252A0A03UFQ87
32 1 (O) 0 136.732 GB DB02 TOSHIBA MK1401GRRB DB0252B0A034FQ87
32 2 (O) 0 931.512 GB AS06 SEAGATE ST91000640SS AS069XG2TQ6K
32 3 (O) 0 931.512 GB AS06 SEAGATE ST91000640SS AS069XG2SDVB
Adapter 1 (PERC H810 Adapter) enclosures found: 1
Adapter 1 (PERC H810 Adapter) enclosure 63 slots found: 15
Encl Slot State P.Fail.Count Raw Size Drive FW Inquiry Data
63 0 (OS) 0 186.310 GB C190 HITACHI HUSRL402 CLAR200C190XTVUNYRA
63 1 (OS) 0 186.310 GB C190 HITACHI HUSRL402 CLAR200C190XTVTK14A
63 2 (OS) 0 186.310 GB C190 HITACHI HUSRL402 CLAR200C190XTVUY4UA
63 3 (OS) 0 186.310 GB C190 HITACHI HUSRL402 CLAR200C190XTVUY8RA
63 4 (OS) 0 186.310 GB C190 HITACHI HUSRL402 CLAR200C190XTVTZR7A
63 5 (US) 0 366.796 GB C342 HITACHI HUSMM144 CLAR400C3420QY5DU8A
63 6 (OS) 0 186.310 GB C190 HITACHI HUSRL402 CLAR200C190XTVU1TWA
63 7 (O) 0 1.819 TB C1D6 HITACHI HUS72302CLAR2000C1D6YGKT4W2K
63 8 (O) 0 1.819 TB C1D6 HITACHI HUS72302CLAR2000C1D6YGKSVK4K
63 9 (O) 0 1.819 TB C1D6 HITACHI HUS72302CLAR2000C1D6YGKT9SYK
63 10 (O) 0 1.819 TB C1D6 HITACHI HUS72302CLAR2000C1D6YGKSXDYK
63 11 (O) 0 1.819 TB C1D6 HITACHI HUS72302CLAR2000C1D6YGKSXS4K
63 12 (O) 0 1.819 TB C1D6 HITACHI HUS72302CLAR2000C1D6YGKSTB7K
63 13 (O) 0 1.819 TB C1D6 HITACHI HUS72302CLAR2000C1D6YGKSXBBK
63 14 (GEI) 0 1.819 TB C1D6 HITACHI HUS72302CLAR2000C1D6YGKSXW6K Hotspare Information
WARNING: Physical disk problems have been found.
-------------------
Adapter Information
-------------------
Adapters present: 2
Adapter 0:
[Product Name] PERC H710P Mini
[Firmware Version] 3.130.05-1796
[Serial Number] 2B502VA
[Memory] 1024MB
[Alarm Device] Absent
[Alarm Enabled] Disabled
[Memory Correctable Errors] 0
[Memory Uncorrectable Errors] 0
[Battery Backup Unit] Present
[Restore HotSpare on Insertion] Disabled
[Use Full Disk Encryption Only] Yes
[Security Keys Assigned] No
[Security Keys Failed] No
[Security Keys Not Backed Up] No
[Virtual Drives] 2
[VD's Degraded] 0
[VD's Offline] 0
[Failed Disks] 0
Adapter 1:
[Product Name] PERC H810 Adapter
[Firmware Version] 3.130.05-1796
[Serial Number] 2480034
[Memory] 1024MB
[Alarm Device] Absent
[Alarm Enabled] Disabled
[Memory Correctable Errors] 0
[Memory Uncorrectable Errors] 0
[Battery Backup Unit] Present
[Restore HotSpare on Insertion] Disabled
[Use Full Disk Encryption Only] Yes
[Security Keys Assigned] No
[Security Keys Failed] No
[Security Keys Not Backed Up] No
[Virtual Drives] 2
[VD's Degraded] 1
[VD's Offline] 0
[Failed Disks] 0
SSD enclosure 63 slot 5 is showing "US"
may proceed with this below command
/opt/MegaRAID/MegaCli/MegaCli64 -PDOnline -PhysDrv[63:5] -a1
or this SSD having hardware failure issue
i am littile bit confuse ,please guide me.
2023-11-14 09:20 AM
Parichay_Saxena,
As it is a RAID 5 the new SSD will not automatically be pulled into the array. The reason it is showing a hardware failure is just because the SSD is sitting there as a Unconfigured Good drive. Once it is added into the SSD array that error will no longer appear.
There are two options available to you.
1. Set the SSD as a hot spare and the array may pull it in automatically. However, given that the hard drive is a larger size than the others it may not do it. Just know that the larger hard drive will work with the array, you just won't have complete access to all the space on the larger drive as it will limit it to the size of the other SSDs.
2. Manually add the drive back into the array and initiate a rebuild of the disk from the other SSDs.
If you want to try the hot spare way, here are the steps. After these steps I will also provide the steps for adding in the drive manually.
Hot Spare Instructions
1. /opt/MegaRAID/MegaCli/MegaCli64 -PDHSP - Set -PhysDrv[63:5] -a1
2. Once this is set the array should notice there is a drive missing and pull the hard drive in and begin the rebuild process.
Manually adding the drive into the array
1. /opt/MegaRAID/MegaCli/MegaCli64 - PdGetMissing -aALL
This will show the array and row of the missing drive as you will need it for the next step
2. /opt/MegaRAID/MegaCli/MegaCli64 -PdReplaceMissing -PhysDrv[63:5] -Array<array number from step 1> -row<row number from step 1> -a1
This should add the drive back into the array at the location that the old drive occupied.
3. /opt/MegaRAID/MegaCli/MegaCli64 -PDRbld -Start -PhysDrv[63:5] -a1
This will start the rebuild process to recreate the missing disk from the parity information found on the other SSDs.
4. /opt/MegaRAID/MegaCli/MegaCli64 -PDRbld -ShowProg -PhysDrv[63:5] -a1
This shows the rebuild progress. This can take some time as the adapter deprioritizes array rebuilds compared to normal traffic.
I hope these two options help.
2023-11-20 08:02 PM
Based on everything I can find you may have a couple of options that may or may not work here.
1. Upgrade the PERC H810 firmware to whatever the latest version you can find. It is possible that the firmware does not allow for the different revisions that you are seeing on the blocked drive compared to the one that is working.
2. See if there is a firmware update for the blocked drive on the Hitachi site to bring it up to the revision that matches the other Hitachi drives.
3. The sector sizes may not be the same between the two Hitachi hard drives. If they aren't the same this can block the drive as well. We have seen this with drives from Dell/EMC where the drive was low level formatted for a SAN. SAN formatted drives use a different sector size than normal Linux/Windows servers utilize. You should be able to see all the drive information when you run the nwraidtool script and compare all the information between the blocked drive and the working drives.
I'm sorry but this is all I can think of at this time that may cause the blocking that you are seeing. I suspect it is the Drive Firmware (Revision) that is probably the issue but a lot of people on the internet got around blocked drive issues with a controller card firmware update.
I hope this helps.
2023-11-14 09:20 AM
Parichay_Saxena,
As it is a RAID 5 the new SSD will not automatically be pulled into the array. The reason it is showing a hardware failure is just because the SSD is sitting there as a Unconfigured Good drive. Once it is added into the SSD array that error will no longer appear.
There are two options available to you.
1. Set the SSD as a hot spare and the array may pull it in automatically. However, given that the hard drive is a larger size than the others it may not do it. Just know that the larger hard drive will work with the array, you just won't have complete access to all the space on the larger drive as it will limit it to the size of the other SSDs.
2. Manually add the drive back into the array and initiate a rebuild of the disk from the other SSDs.
If you want to try the hot spare way, here are the steps. After these steps I will also provide the steps for adding in the drive manually.
Hot Spare Instructions
1. /opt/MegaRAID/MegaCli/MegaCli64 -PDHSP - Set -PhysDrv[63:5] -a1
2. Once this is set the array should notice there is a drive missing and pull the hard drive in and begin the rebuild process.
Manually adding the drive into the array
1. /opt/MegaRAID/MegaCli/MegaCli64 - PdGetMissing -aALL
This will show the array and row of the missing drive as you will need it for the next step
2. /opt/MegaRAID/MegaCli/MegaCli64 -PdReplaceMissing -PhysDrv[63:5] -Array<array number from step 1> -row<row number from step 1> -a1
This should add the drive back into the array at the location that the old drive occupied.
3. /opt/MegaRAID/MegaCli/MegaCli64 -PDRbld -Start -PhysDrv[63:5] -a1
This will start the rebuild process to recreate the missing disk from the parity information found on the other SSDs.
4. /opt/MegaRAID/MegaCli/MegaCli64 -PDRbld -ShowProg -PhysDrv[63:5] -a1
This shows the rebuild progress. This can take some time as the adapter deprioritizes array rebuilds compared to normal traffic.
I hope these two options help.
2023-11-17 06:30 PM
@JohnKisner thank you so much.
2023-11-20 09:36 AM
You're welcome.
2023-11-20 06:02 PM - edited 2023-11-20 06:05 PM
We replaced SSD with same capacity. but SSD is showing blocked and now not
able to configure in VD group.
Please refer to enclosed snapshots.
2023-11-20 08:02 PM
Based on everything I can find you may have a couple of options that may or may not work here.
1. Upgrade the PERC H810 firmware to whatever the latest version you can find. It is possible that the firmware does not allow for the different revisions that you are seeing on the blocked drive compared to the one that is working.
2. See if there is a firmware update for the blocked drive on the Hitachi site to bring it up to the revision that matches the other Hitachi drives.
3. The sector sizes may not be the same between the two Hitachi hard drives. If they aren't the same this can block the drive as well. We have seen this with drives from Dell/EMC where the drive was low level formatted for a SAN. SAN formatted drives use a different sector size than normal Linux/Windows servers utilize. You should be able to see all the drive information when you run the nwraidtool script and compare all the information between the blocked drive and the working drives.
I'm sorry but this is all I can think of at this time that may cause the blocking that you are seeing. I suspect it is the Drive Firmware (Revision) that is probably the issue but a lot of people on the internet got around blocked drive issues with a controller card firmware update.
I hope this helps.