Topic: RAID 1 Hot spare indicator?

[smiit]

Preparing to migrate from SME 8 to 9 and uncertain about RAID 1 status in the console message.

Default install from DVD to a Dell PowerEdge T310 with three 2TB SATA drives in front hot-swap bay.

After install, updates and RAID resync I see:

Code: [Select]

# cat /proc/mdstat
Personalities : [raid1]
md0 : active raid1 sdc1[2] sdb1[1] sda1[0]
      255936 blocks super 1.0 [3/3] [UUU]

md1 : active raid1 sdc2[2] sda2[0] sdb2[1]
      1953126208 blocks super 1.1 [3/3] [UUU]
      bitmap: 0/15 pages [0KB], 65536KB chunk

unused devices: <none>

And:

Code: [Select]

# mdadm --detail /dev/md1
/dev/md1:
        Version : 1.1
  Creation Time : Thu Dec 18 06:15:49 2014
     Raid Level : raid1
     Array Size : 1953126208 (1862.65 GiB 2000.00 GB)
  Used Dev Size : 1953126208 (1862.65 GiB 2000.00 GB)
   Raid Devices : 3
  Total Devices : 3
    Persistence : Superblock is persistent

  Intent Bitmap : Internal

    Update Time : Tue Dec 23 15:29:22 2014
          State : active
 Active Devices : 3
Working Devices : 3
 Failed Devices : 0
  Spare Devices : 0

           Name : localhost.localdomain:1
           UUID : xxxxxxxx:xxxxxxxx:xxxxxxxx:xxxxxxxx
         Events : 4689

    Number   Major   Minor   RaidDevice State
       0       8        2        0      active sync   /dev/sda2
       1       8       18        1      active sync   /dev/sdb2
       2       8       34        2      active sync   /dev/sdc2

I assume it loaded as a RAID 1 with a hot spare since the total drive space is 2TB (less overhead) but I would expect this:

Code: [Select]

Active Devices : 3
Working Devices : 3
 Failed Devices : 0
  Spare Devices : 0

to appear as:

Code: [Select]

Active Devices : 3
Working Devices : 2
Failed Devices : 0
Spare Devices : 1

Is this the standard output for a clean 3 drive RAID1 setup in the SME 9 environment?

Or should I reinstall with different settings to have the desired two-drive RAID 1 with an available hot-spare?

Thanks

[nicolatiana]

The spare handling for RAID arrays is not implemented.

(From release notes of SME9)
I suppose is not supported in install, but performing install with two drives and then adding the third, with mdadm I suppose you can add it as spare to the array.
Can anyone confirm this ?

Nicola

[stephdl]

for adding a spare drive you can follow this howto http://wiki.contribs.org/Raid#Adding_another_Hard_Drive_Later_.28Raid1_array_only.29

the easy way is to reinstall, but i'm not sure that you need it, probably you can use mdadm to --remove and --set-faulty the drive you want of raid partitions (/dev/md0 & /dev/md1) and after add the correct number of disk in /etc/mdadm.conf

Code: [Select]

mdadm --set-faulty /dev/md0 /dev/sdc1
mdadm --set-faulty /dev/md1 /dev/sdc2
mdadm --remove /dev/md0 /dev/sdc1
mdadm --remove /dev/md1 /dev/sdc2
and

Code: [Select]

# cat /etc/mdadm.conf
# mdadm.conf written out by anaconda
MAILADDR root
AUTO +imsm +1.x -all
ARRAY /dev/md0 level=raid1 num-devices=2 UUID=5ee074e0:f3fa7c60:27383b37:0663f6f2
ARRAY /dev/md1 level=raid1 num-devices=2 UUID=207ed099:c5abe949:0dc10494:758f4087

then follow up the pointer I gave, of course I assumed that all backups of your data are done.

[smiit]

for adding a spare drive you can follow this howto http://wiki.contribs.org/Raid#Adding_another_Hard_Drive_Later_.28Raid1_array_only.29

the easy way is to reinstall, but i'm not sure that you need it, probably you can use mdadm to --remove and --set-faulty the drive you want of raid partitions (/dev/md0 & /dev/md1) and after add the correct number of disk in /etc/mdadm.conf

then follow up the pointer I gave, of course I assumed that all backups of your data are done.

Thank you, stephdl, for the info.

I followed your suggestions and failed/removed /dev/sdc from the array and added it back in but it still appears exactly the same as an active/working device, not a spare (which is what I keep expecting/desire) when I run the mdadm --detail command.

Code: [Select]

Active Devices : 3
Working Devices : 3
Failed Devices : 0
Spare Devices : 0

If I re-install is there a specific RAID installation option I need to specify that I missed before?

Also, I did see Nicola's comment about spare handling not implemented in the release notes .. is there any 'best practices' advice for what to do currently in a production environment?

Should I install 2 drives as a standard RAID1 mirror and just have the spare ready to drop in if necessary?  Or will that 3rd drive effectively act as a hot spare if one fails?

[nicolatiana]

Did you wipe-out with dd the disk before putting in back ?

Nicola

[smiit]

Did you wipe-out with dd the disk before putting in back ?

Nicola

I did not.

Tried the same sequence again but wiped /dev/sdc with the following dd command:

Code: [Select]

dd if=/dev/zero of=/dev/sdc bs=512 count=1

Rebooted and copied partition table:

Code: [Select]

sfdisk -d /dev/sda > sfdisk_sda.output
sfdisk /dev/sdc < sfdisk_sda.output

Rebooted again but now I get this error:

Code: [Select]

mdadm --add /dev/md0 /dev/sdc1
mdadm: Cannot find /dev/sdc1: No such file or directory


[stephdl]

Code: [Select]

# cat /etc/mdadm.conf
# mdadm.conf written out by anaconda
MAILADDR root
AUTO +imsm +1.x -all
ARRAY /dev/md0 level=raid1 num-devices=2 UUID=5ee074e0:f3fa7c60:27383b37:0663f6f2
ARRAY /dev/md1 level=raid1 num-devices=2 UUID=207ed099:c5abe949:0dc10494:758f4087


do you have modified the /etc/mdadm.conf, ok this was before you wiped the drive

[nicolatiana]

Output of fdsik -l ?

Nicola

[smiit]

Output of fdsik -l ?

Nicola

Code: [Select]

# fdisk -l

Disk /dev/sdb: 2000.4 GB, 2000398934016 bytes
255 heads, 63 sectors/track, 243201 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00096293

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1   *           1          32      256000   fd  Linux raid autodetect
Partition 1 does not end on cylinder boundary.
/dev/sdb2              32      243202  1953257472   fd  Linux raid autodetect

Disk /dev/sdc: 2000.4 GB, 2000398934016 bytes
255 heads, 63 sectors/track, 243201 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000


Disk /dev/sda: 2000.4 GB, 2000398934016 bytes
255 heads, 63 sectors/track, 243201 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0006f267

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          32      256000   fd  Linux raid autodetect
Partition 1 does not end on cylinder boundary.
/dev/sda2              32      243202  1953257472   fd  Linux raid autodetect

Disk /dev/md1: 2000.0 GB, 2000001236992 bytes
2 heads, 4 sectors/track, 488281552 cylinders
Units = cylinders of 8 * 512 = 4096 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000


Disk /dev/mapper/main-root: 1991.6 GB, 1991568785408 bytes
255 heads, 63 sectors/track, 242127 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000


Disk /dev/mapper/main-swap: 8430 MB, 8430551040 bytes
255 heads, 63 sectors/track, 1024 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000


Disk /dev/md0: 262 MB, 262078464 bytes
2 heads, 4 sectors/track, 63984 cylinders
Units = cylinders of 8 * 512 = 4096 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

And

Code: [Select]

cat /etc/mdadm.conf
# mdadm.conf written out by anaconda
MAILADDR root
AUTO +imsm +1.x -all
ARRAY /dev/md0 level=raid1 num-devices=2 UUID= xxx
ARRAY /dev/md1 level=raid1 num-devices=2 UUID= xxx


[nicolatiana]

No partitions in /dev/sdc

Nicola

[janet]

smiit

Just reinstall with only 2 physical drives attached, clean them with the dd command first.
Then just keep the third drive as a spare that you can connect to a SATA port later & auto/manually add to the array using the Console menu, if & when needed.

You may waste a lot of time troubleshooting the 3 drive array.

[nicolatiana]

Then just keep the third drive as a spare that you can connect to a SATA port later & auto/manually add to the array using the Console menu, if & when needed.
You may waste a lot of time troubleshooting the 3 drive array.

Janet, this means the "hot spare" is not supported at all in SME 9 ? (both in install and in runtime).

Nicola

[stephdl]

You should able to add a drive as hot spare by the CL, but like you noticed there is no partition in /dev/sdc, that could be a start for troubleshoot the issue.
We didn't spend time on the hot spare during the sme9 development mainly because it is the first things that a sysadmin wants to disable...usually.

[janet]

nicolatiana (& smiit et al)

this means the "hot spare" is not supported at all in SME 9 ? (both in install and in runtime).

See the Notes section of SME 9 Release notes at top of this Forum.
http://forums.contribs.org/index.php/topic,51133.msg259082.html#msg259082
IIUIC as per the release notes, the installer does not configure the RAID with a hot standby "spare".

In my earlier post, I just meant that it is easy to add a drive & resync it if one RAID1 drive fails, rather than spend considerable effort working out how to add a hot spare drive.
It would be a very rare occasion that 2 drives would fail at the same time, so there is usually some time available (while running in a single drive degraded array mode), in which to replace a faulty drive.

.....Although I have had both drives of a RAID1 software array fail at the sane time (each failed in a different way) & that was quite distressing at the time, but in that case you need to rely on a good backup, so daily backups are VERY IMPORTANT.

If a admin user physically connects a third drive & somehow tries to configure a RAID1 spare drive during install, then they may have difficulties unravelling the drive configuration.
IMO it is better to install SME as a 2 drive RAID1 array (without a third drive being physically connected) , & after reboot & syncing have completed, you could then follow a generic Linux CentOS Howto to manually add a third hot spare drive to the RAID1 array.
SME Server is a flavour of Linux & you can do just about anything with Linux if you know how.

stephdl's comments also support this ie add a spare using CLI.

[stephdl]

IMHO if you have 3 drives, then you could do a Raid5 and get more drive space.