Koozali.org: home of the SME Server
Obsolete Releases => SME Server 7.x => Topic started by: engdev on June 27, 2011, 01:14:35 PM
-
Hi,
I recently installed 7.5.1 on a new server (HP ML 110 G6) with 4 identical HDD and followed the 'Moving SME to new hardware' howto to transfer everything from our existing server (also with 4 HDD but started out with 7.3 ? and upgraded to 7.5.1).
Last week I received A DegradedArray event has been detected on md device /dev/md1
so I removed & re-added the sda1 to the array and then received A SpareActive event has been detected on md device /dev/md1.
Device /dev/sda1 is now an active member of md device /dev/md1
This is the output of mdadm.conf and mdstat:
DEVICE partitions
ARRAY /dev/md2 level=raid5 num-devices=3 spares=2 UUID=85f95d35:6ee44c78:8bb85570:19e5f164
ARRAY /dev/md1 level=raid1 num-devices=4 UUID=58368727:fab34221:15b334b3:7232af96
[root@smeserver ~]# cat /proc/mdstat
Personalities : [raid1] [raid5]
md2 : active raid5 sdc2[0] sdd2[2] sdb2[1]
976558592 blocks level 5, 256k chunk, algorithm 2 [3/3] [UUU]
md1 : active raid1 sda1[0] sdd1[3] sdc1[2] sdb1[1]
104320 blocks [4/4] [UUUU]
unused devices: <none>
However, the output of mdadm.conf and mdstat on the original server is as follows:
DEVICE partitions
ARRAY /dev/md2
ARRAY /dev/md1
[root@smeserver3 ~]# cat /proc/mdstat
Personalities : [raid1] [raid5]
md2 : active raid5 sda2[0] sdc2[3] sdd2[2] sdb2[1]
490014208 blocks level 5, 256k chunk, algorithm 2 [3/3] [UUU]
md1 : active raid1 sda1[0] sdd1[3] sdc1[2] sdb1[1]
104320 blocks [3/3] [UUU]
unused devices: <none>
So on the old server, sda2 was part of the Raid 5 array but on the new server it is missing. Is this due to sda1 being kicked out of the array on the new server when the array degraded? Do I need to add sda2 back into the md2 array?
Is the Raid 5 array on the new server configured correctly or do I need to take any other action?
How can you tell from the mdstat if the Raid 5 array & spare is functioning properly?
Thanks for any advice (& thanks for the great software)
(Apologies if I am repeating a previous question but I had a search here and on Google and couldn't find anything that matched my installation with 4 disks.)
-
The RAID 5 is "working properly" but as a 3 disk RAID array. The 4th disk (sda2) is missing.
A quick check is to use the df command (df -h) to show the disk space available.
For RAID5, the space available (in md2) should be approximately 3 x (single disk size) less 10% or so.
So a 4 disk array, using 500 Gb disks should give you about 1400 Gb
I think when you try it, you will get a value nearer to 2 x (single disk size)
The RAID 5 rule of thumb is : disk space = (Number of disks -1) x (single disk size)
Sorry, but I can't help you with getting the 4th disk into the array since I am not sure how to do that on a "live"system.
Cheers
Ian
-
Thanks very much for the df -h tip Ian, it's very useful information.
I agree with your calc for RAID 5 nospare but 'I think' the default SME RAID5 (as I have) is to have (n-1) x capacity plus a spare so in my case with 4 x 500 Gb the total is approx 1Tb, which is what is shown by df -h.
I found another mdadm command:
mdadm --examine /dev/md2 /dev/sda2which returns these results:
mdadm: No super block found on /dev/md2 (Expected magic a92b4efc, got 00000000)
/dev/sda2:
Magic : a92b4efc
Version : 00.90.00
UUID : 85f95d35:6ee44c78:8bb85570:19e5f164
Creation Time : Fri May 6 13:14:17 2011
Raid Level : raid5
Raid Devices : 3
Total Devices : 4
Preferred Minor : 2
Update Time : Mon Jun 6 21:12:52 2011
State : clean
Active Devices : 3
Working Devices : 4
Failed Devices : 0
Spare Devices : 1
Checksum : fc906df3 - correct
Events : 0.836279
Layout : left-symmetric
Chunk Size : 256K
Number Major Minor RaidDevice State
this 0 8 2 0 active sync /dev/sda2
0 0 8 2 0 active sync /dev/sda2
1 1 8 18 1 active sync /dev/sdb2
2 2 8 50 2 active sync /dev/sdd2
3 3 8 34 3 spare /dev/sdc2
Which indicates that the array is correct but it contradicts the results from cat /proc/mdstat which still shows sda2 missing from the array. So I am still confused, if you or anyone else can educate me as to why there's a difference between mdstat and the mdadm --examine, I would be very grateful.
regards
John
-
Yes, the output clearly says it is RAID 5 with 3 devices....with a "hot spare". This means if a disk fails, the "hot spare" can be swapped in to take its place with minimum downtime.
If you want all 4 disks to be used to give extra storage (ie RAID 5 with 4-disk array, no spare) you install using the "sme nospare" option at the beginning of the installation - the very first thing you do after the installation CD had loaded the installer and presented you with an SME installation screen.
Can you re-install? If so, use "sme nospare" at the beginning of the installation. That is the command to make SME use all the available disks in the RAID 5 array.
Search the Wiki and the forums for advice on this.
Cheers
Ian