Koozali.org: home of the SME Server

RAID 1 Hot spare indicator?

Offline smiit

  • ***
  • 41
  • +0/-0
RAID 1 Hot spare indicator?
« on: December 23, 2014, 10:39:23 PM »
Preparing to migrate from SME 8 to 9 and uncertain about RAID 1 status in the console message.

Default install from DVD to a Dell PowerEdge T310 with three 2TB SATA drives in front hot-swap bay.

After install, updates and RAID resync I see:

Code: [Select]
# cat /proc/mdstat
Personalities : [raid1]
md0 : active raid1 sdc1[2] sdb1[1] sda1[0]
      255936 blocks super 1.0 [3/3] [UUU]

md1 : active raid1 sdc2[2] sda2[0] sdb2[1]
      1953126208 blocks super 1.1 [3/3] [UUU]
      bitmap: 0/15 pages [0KB], 65536KB chunk

unused devices: <none>

And:

Code: [Select]
# mdadm --detail /dev/md1
/dev/md1:
        Version : 1.1
  Creation Time : Thu Dec 18 06:15:49 2014
     Raid Level : raid1
     Array Size : 1953126208 (1862.65 GiB 2000.00 GB)
  Used Dev Size : 1953126208 (1862.65 GiB 2000.00 GB)
   Raid Devices : 3
  Total Devices : 3
    Persistence : Superblock is persistent

  Intent Bitmap : Internal

    Update Time : Tue Dec 23 15:29:22 2014
          State : active
 Active Devices : 3
Working Devices : 3
 Failed Devices : 0
  Spare Devices : 0

           Name : localhost.localdomain:1
           UUID : xxxxxxxx:xxxxxxxx:xxxxxxxx:xxxxxxxx
         Events : 4689

    Number   Major   Minor   RaidDevice State
       0       8        2        0      active sync   /dev/sda2
       1       8       18        1      active sync   /dev/sdb2
       2       8       34        2      active sync   /dev/sdc2

I assume it loaded as a RAID 1 with a hot spare since the total drive space is 2TB (less overhead) but I would expect this:

Code: [Select]
Active Devices : 3
Working Devices : 3
 Failed Devices : 0
  Spare Devices : 0

to appear as:

Code: [Select]
Active Devices : 3
Working Devices : 2
Failed Devices : 0
Spare Devices : 1

Is this the standard output for a clean 3 drive RAID1 setup in the SME 9 environment?

Or should I reinstall with different settings to have the desired two-drive RAID 1 with an available hot-spare?

Thanks

Offline nicolatiana

  • *
  • 724
  • +0/-0
Re: RAID 1 Hot spare indicator?
« Reply #1 on: December 24, 2014, 01:27:21 AM »
Quote
The spare handling for RAID arrays is not implemented.
(From release notes of SME9)
I suppose is not supported in install, but performing install with two drives and then adding the third, with mdadm I suppose you can add it as spare to the array.
Can anyone confirm this ?

Nicola
Consulente di Smeserver.it -  Soluzioni e supporto su Sme server in Italia.

Offline stephdl

  • *
  • 1,523
  • +0/-0
    • Linux et Geekeries
Re: RAID 1 Hot spare indicator?
« Reply #2 on: December 24, 2014, 07:00:10 AM »
for adding a spare drive you can follow this howto http://wiki.contribs.org/Raid#Adding_another_Hard_Drive_Later_.28Raid1_array_only.29

the easy way is to reinstall, but i'm not sure that you need it, probably you can use mdadm to --remove and --set-faulty the drive you want of raid partitions (/dev/md0 & /dev/md1) and after add the correct number of disk in /etc/mdadm.conf

Code: [Select]
mdadm --set-faulty /dev/md0 /dev/sdc1
mdadm --set-faulty /dev/md1 /dev/sdc2
mdadm --remove /dev/md0 /dev/sdc1
mdadm --remove /dev/md1 /dev/sdc2

and

Code: [Select]
# cat /etc/mdadm.conf
# mdadm.conf written out by anaconda
MAILADDR root
AUTO +imsm +1.x -all
ARRAY /dev/md0 level=raid1 num-devices=2 UUID=5ee074e0:f3fa7c60:27383b37:0663f6f2
ARRAY /dev/md1 level=raid1 num-devices=2 UUID=207ed099:c5abe949:0dc10494:758f4087

then follow up the pointer I gave, of course I assumed that all backups of your data are done.
See http://wiki.contribs.org/Koozali_Foundation
irc : Freenode #sme_server #sme-fr

!!! Please write your knowledge to the Wiki !!!

Offline smiit

  • ***
  • 41
  • +0/-0
Re: RAID 1 Hot spare indicator?
« Reply #3 on: December 24, 2014, 04:01:58 PM »
for adding a spare drive you can follow this howto http://wiki.contribs.org/Raid#Adding_another_Hard_Drive_Later_.28Raid1_array_only.29

the easy way is to reinstall, but i'm not sure that you need it, probably you can use mdadm to --remove and --set-faulty the drive you want of raid partitions (/dev/md0 & /dev/md1) and after add the correct number of disk in /etc/mdadm.conf

then follow up the pointer I gave, of course I assumed that all backups of your data are done.

Thank you, stephdl, for the info.

I followed your suggestions and failed/removed /dev/sdc from the array and added it back in but it still appears exactly the same as an active/working device, not a spare (which is what I keep expecting/desire) when I run the mdadm --detail command.

Code: [Select]
Active Devices : 3
Working Devices : 3
Failed Devices : 0
Spare Devices : 0

If I re-install is there a specific RAID installation option I need to specify that I missed before?

Also, I did see Nicola's comment about spare handling not implemented in the release notes .. is there any 'best practices' advice for what to do currently in a production environment?

Should I install 2 drives as a standard RAID1 mirror and just have the spare ready to drop in if necessary?  Or will that 3rd drive effectively act as a hot spare if one fails?

« Last Edit: December 24, 2014, 04:14:13 PM by smiit »

Offline nicolatiana

  • *
  • 724
  • +0/-0
Re: RAID 1 Hot spare indicator?
« Reply #4 on: December 24, 2014, 04:42:50 PM »
Did you wipe-out with dd the disk before putting in back ?

Nicola
Consulente di Smeserver.it -  Soluzioni e supporto su Sme server in Italia.

Offline smiit

  • ***
  • 41
  • +0/-0
Re: RAID 1 Hot spare indicator?
« Reply #5 on: December 24, 2014, 05:52:47 PM »
Did you wipe-out with dd the disk before putting in back ?

Nicola

I did not.

Tried the same sequence again but wiped /dev/sdc with the following dd command:

Code: [Select]
dd if=/dev/zero of=/dev/sdc bs=512 count=1

Rebooted and copied partition table:

Code: [Select]
sfdisk -d /dev/sda > sfdisk_sda.output
sfdisk /dev/sdc < sfdisk_sda.output

Rebooted again but now I get this error:

Code: [Select]
mdadm --add /dev/md0 /dev/sdc1
mdadm: Cannot find /dev/sdc1: No such file or directory


Offline stephdl

  • *
  • 1,523
  • +0/-0
    • Linux et Geekeries
Re: RAID 1 Hot spare indicator?
« Reply #6 on: December 24, 2014, 06:54:49 PM »
Code: [Select]
# cat /etc/mdadm.conf
# mdadm.conf written out by anaconda
MAILADDR root
AUTO +imsm +1.x -all
ARRAY /dev/md0 level=raid1 num-devices=2 UUID=5ee074e0:f3fa7c60:27383b37:0663f6f2
ARRAY /dev/md1 level=raid1 num-devices=2 UUID=207ed099:c5abe949:0dc10494:758f4087

do you have modified the /etc/mdadm.conf, ok this was before you wiped the drive
See http://wiki.contribs.org/Koozali_Foundation
irc : Freenode #sme_server #sme-fr

!!! Please write your knowledge to the Wiki !!!

Offline nicolatiana

  • *
  • 724
  • +0/-0
Re: RAID 1 Hot spare indicator?
« Reply #7 on: December 24, 2014, 06:57:12 PM »
Output of fdsik -l ?

Nicola
Consulente di Smeserver.it -  Soluzioni e supporto su Sme server in Italia.

Offline smiit

  • ***
  • 41
  • +0/-0
Re: RAID 1 Hot spare indicator?
« Reply #8 on: December 24, 2014, 07:44:15 PM »
Output of fdsik -l ?

Nicola

Code: [Select]
# fdisk -l

Disk /dev/sdb: 2000.4 GB, 2000398934016 bytes
255 heads, 63 sectors/track, 243201 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00096293

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1   *           1          32      256000   fd  Linux raid autodetect
Partition 1 does not end on cylinder boundary.
/dev/sdb2              32      243202  1953257472   fd  Linux raid autodetect

Disk /dev/sdc: 2000.4 GB, 2000398934016 bytes
255 heads, 63 sectors/track, 243201 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000


Disk /dev/sda: 2000.4 GB, 2000398934016 bytes
255 heads, 63 sectors/track, 243201 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0006f267

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          32      256000   fd  Linux raid autodetect
Partition 1 does not end on cylinder boundary.
/dev/sda2              32      243202  1953257472   fd  Linux raid autodetect

Disk /dev/md1: 2000.0 GB, 2000001236992 bytes
2 heads, 4 sectors/track, 488281552 cylinders
Units = cylinders of 8 * 512 = 4096 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000


Disk /dev/mapper/main-root: 1991.6 GB, 1991568785408 bytes
255 heads, 63 sectors/track, 242127 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000


Disk /dev/mapper/main-swap: 8430 MB, 8430551040 bytes
255 heads, 63 sectors/track, 1024 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000


Disk /dev/md0: 262 MB, 262078464 bytes
2 heads, 4 sectors/track, 63984 cylinders
Units = cylinders of 8 * 512 = 4096 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

And

Code: [Select]
cat /etc/mdadm.conf
# mdadm.conf written out by anaconda
MAILADDR root
AUTO +imsm +1.x -all
ARRAY /dev/md0 level=raid1 num-devices=2 UUID= xxx
ARRAY /dev/md1 level=raid1 num-devices=2 UUID= xxx


Offline nicolatiana

  • *
  • 724
  • +0/-0
Re: RAID 1 Hot spare indicator?
« Reply #9 on: December 24, 2014, 09:33:55 PM »
No partitions in /dev/sdc

Nicola
Consulente di Smeserver.it -  Soluzioni e supporto su Sme server in Italia.

Offline janet

  • *****
  • 4,812
  • +0/-0
Re: RAID 1 Hot spare indicator?
« Reply #10 on: December 25, 2014, 01:45:02 AM »
smiit

Just reinstall with only 2 physical drives attached, clean them with the dd command first.
Then just keep the third drive as a spare that you can connect to a SATA port later & auto/manually add to the array using the Console menu, if & when needed.

You may waste a lot of time troubleshooting the 3 drive array.
Please search before asking, an answer may already exist.
The Search & other links to useful information are at top of Forum.

Offline nicolatiana

  • *
  • 724
  • +0/-0
Re: RAID 1 Hot spare indicator?
« Reply #11 on: December 26, 2014, 06:57:05 PM »
Then just keep the third drive as a spare that you can connect to a SATA port later & auto/manually add to the array using the Console menu, if & when needed.
You may waste a lot of time troubleshooting the 3 drive array.
Janet, this means the "hot spare" is not supported at all in SME 9 ? (both in install and in runtime).

Nicola
Consulente di Smeserver.it -  Soluzioni e supporto su Sme server in Italia.

Offline stephdl

  • *
  • 1,523
  • +0/-0
    • Linux et Geekeries
Re: RAID 1 Hot spare indicator?
« Reply #12 on: December 26, 2014, 09:04:48 PM »
You should able to add a drive as hot spare by the CL, but like you noticed there is no partition in /dev/sdc, that could be a start for troubleshoot the issue.
We didn't spend time on the hot spare during the sme9 development mainly because it is the first things that a sysadmin wants to disable...usually.
« Last Edit: December 26, 2014, 09:30:11 PM by stephdl »
See http://wiki.contribs.org/Koozali_Foundation
irc : Freenode #sme_server #sme-fr

!!! Please write your knowledge to the Wiki !!!

Offline janet

  • *****
  • 4,812
  • +0/-0
Re: RAID 1 Hot spare indicator?
« Reply #13 on: December 26, 2014, 11:00:03 PM »
nicolatiana (& smiit et al)

Quote
this means the "hot spare" is not supported at all in SME 9 ? (both in install and in runtime).

See the Notes section of SME 9 Release notes at top of this Forum.
http://forums.contribs.org/index.php/topic,51133.msg259082.html#msg259082
IIUIC as per the release notes, the installer does not configure the RAID with a hot standby "spare".

In my earlier post, I just meant that it is easy to add a drive & resync it if one RAID1 drive fails, rather than spend considerable effort working out how to add a hot spare drive.
It would be a very rare occasion that 2 drives would fail at the same time, so there is usually some time available (while running in a single drive degraded array mode), in which to replace a faulty drive.

.....Although I have had both drives of a RAID1 software array fail at the sane time (each failed in a different way) & that was quite distressing at the time, but in that case you need to rely on a good backup, so daily backups are VERY IMPORTANT.

If a admin user physically connects a third drive & somehow tries to configure a RAID1 spare drive during install, then they may have difficulties unravelling the drive configuration.
IMO it is better to install SME as a 2 drive RAID1 array (without a third drive being physically connected) , & after reboot & syncing have completed, you could then follow a generic Linux CentOS Howto to manually add a third hot spare drive to the RAID1 array.
SME Server is a flavour of Linux & you can do just about anything with Linux if you know how.

stephdl's comments also support this ie add a spare using CLI.
« Last Edit: December 26, 2014, 11:08:56 PM by janet »
Please search before asking, an answer may already exist.
The Search & other links to useful information are at top of Forum.

Offline stephdl

  • *
  • 1,523
  • +0/-0
    • Linux et Geekeries
Re: RAID 1 Hot spare indicator?
« Reply #14 on: December 27, 2014, 12:55:39 AM »
IMHO if you have 3 drives, then you could do a Raid5 and get more drive space.
See http://wiki.contribs.org/Koozali_Foundation
irc : Freenode #sme_server #sme-fr

!!! Please write your knowledge to the Wiki !!!