Topic: HP's hardware RAID vs SME's RAID

[misnerspace]

Hi All,

I was wondering whether any of you could give me some advice on whether to use SME's software RAID or HP's hardware raid would be better for a new server. I have read the following http://wiki.contribs.org/Raid#Raid_Notes.

Over the years SME has treated me very well so I am going to treat it to a brand new HP Proliant ML350 G6 which has an HP Smart Array P410 controller capable of RAID0-5. It will probably have 4 SATA HDs (each 2TB) as we like to have a lot of storage. We tried OpenFiler for additional storage in the past but had a few issues with it in that it didn't work.

Now my questions are this:

is it best to set up each of the 4 disks as RAID 0 and let SME handle the rest

or is it better to group 2 disks to together as RAID 0 so they appear as 1 logical drive to SME. SME will then see two logical drives (each composed of 2 disks) and will then hopefully set up the array as RAID 1.

or do I let HP Smart Array handle it all, it will probably default to RAID5 so that SME will only see 1 disk.

As you can clearly see, I am not an expert but any advice would be really useful.

thanks!

[Stefano]

hi

IMHO raid 0 is E V I L

should one of the hd die, your server is gone and your data lost

use hw raid 5

[misnerspace]

Thanks Stefano for your prompt reply. My gut feeling is also to go HW with RAID5 as well.

If I use 4x 2TB drives, will that mean the server will in effect have 6TB of redundant storage, if I use RAID5?

BTW, I have been using SME for about 5 years now and my lack of knowledge of Linux administration is testament to the simplicity and stability of SME, so thanks to all of you. I am waiting with bated breath for the release of SME 8 as I really think it will have a big impact on the marketplace, particularly since we are entering what history books may one day call the era of austerity!

[CharlieBrady]

Thanks Stefano for your prompt reply. My gut feeling is also to go HW with RAID5 as well.

If you use HW RAID, SME server cannot notify you when a drive has a problem and needs to be replaced. HP has some proprietary add-on software for monitoring the array status. I think there might be flashing lights on the server frontplate as well.

[Stefano]

Charlie is rigth.. AFAIR, as RH (and so CentOS) is supported by HP, there should be a rpm with raid monitoring tools.. take a look into the cd that comes with the server and in HP's site

[fpausp]

Hi,

I have a server with four 2-tb disks and an adaptec hw-raid-controller.
On the adaptec raid-menu I made just 4 normal partitions 50GB each, then I installed sme 7.5

After the installation I created 2tb raid5 arrays with the rest of space .
Now I am using the arrays as a "normal" disk and mount it in the fstab ...

Sme should now recognize it when a disk fail.

[CharlieBrady]

Sme should now recognize it when a disk fail.

No, you assume too much. SME server will not see any I/O errors in the hardware being the RAID5 array.

[fpausp]

No, you assume too much. SME server will not see any I/O errors in the hardware being the RAID5 array.

Yes that´s true, but I mean its half the rent, when I totally lost a hdd I should see it. 

[mwaymouth]

Hi I have used a HP ML350G4P with hardware raid since 2006 and it is very nice. Have set up with HP Smart Array Controller 641 and 5 SCSI disks in Raid 5. Some important points are:
needed to add a battery backup cache unit to the standard 641 Smart Array Controller to do Raid 5
It all worked out of the box with SME no additional drivers were needed. SME sees the array as a single drive.
there are 2 ways to monitor the array. On the front of the server each drive has a light which indicates drive status green ok red failed etc.  So get someone to check regularly
the other way is to use the hp command line utility To monitor array and drive situation  these are some notes from my system. (there will be a later version of hpacucli now)

While server is running
Install rpm hpacucli-7.40-7.linux.rpm    (get from hp web site)
type  hpacucli at root prompt on server
Then type help for options  (there are lots of things you can do with the command line tool)

I use the following command to check my drives

hpacucli controller slot=5 physicaldrive all show

you can either log in and run hpacucli which will report disk and array status which is what i do or better would be to setup a cron job to do this at regular intervals and email the result.

you can add extra drives to the array later but it is not a 5 minute job and I would try and get drive capacity that will last the life of the server from day one.  You need at least 4. 3 for raid 5 plus one for the hot spare. 
A hot swap spare in the hp array is essential so if a drive does fail the array will bring in the spare without you needing to do anything.

you get the HP array set up using the HP Smart Start  CD then install SME when you have the array configured how you want to.

If a drive does fail all you do is unplug it and put a new one it. The array controller will do everything dont even need to power server off.  Its a good idea to buy an identical  spare drive at the same time you do the setup so when you have a failure in year 3 there is no problem getting a matching drive.  I used hp drives which are dearer but there is no warrenty problem having 3rp party drives in a hp box.  So you have a hot spare in the server and a cold spare in the cupboard.  If you never need it thats great. if you do you will be very happy you have one ready to go. 

good luck

[misnerspace]

Hi mwaymouth, thanks for the info, it is very much appreciated. Based on your advice, I will be buying an additional hot and cold spare. SME seems to be very reliable, it will be good to ensure that the hardware is configured to be as reliable as it can be.

[mwaymouth]

Hi glad to be of help. Two more bits of information re the hp server.
The only problem we have had was a dead power supply in year 2. it was under warrenty but hp took 3 days to get the right part to me. We are in new zealand.  On day 1  I diagnosed the problem ie server stone dead and hp India (where all our support and I think also yours) comes from organised to have a new PS shipped from the main supply center in new zealand about 6 hours drive from me. 
Day 2 the part arrives - it is a front bezel for the server not a power supply. Ring hp and hit the roof. 
Day 3 the right part arrives, plug it in and we are up and running again but it spoiled my 100% up time record of some 6 years with that client with SME.

So the outcome of all this was we bought a second power supply.
One of the benefits of getting an ML 350 is you can install 2 hot swap power supplies. They just plug in like the hard drives do and if one fails the other will keep you going. Its worth considering depending how critical it is to you to loose a day or 2 up time and how far you are from hp supplies. There is a geen light on each unit so you know it is ok.

the other thing to share is,  like you I started with minimum drives. 
we had 3 * 72 gb scsi drives running raid 5 giving us 140 gb which seemed heaps as at the time of installation we were replacing a 40GB server.  A couple of years later when we got up around 100 Gb of data I decided to add another drive. Easy I thought with this hp smart array and raid 5.  Turned out to be not so easy

Firstly  although the hp controller I had would do raid 5 ok it turns out to add another physical drive to the array you must go through 2 processes. Array Expansion and array Extension.  You cant do this and keep all your data intact without having the optional battery backup cache on the smart array.  So we got the battery backup cache unit and I am ready to do the expansion / extension process. 

You should check if your controller has this battery back up on it.

The Extension and Expansion process is done buy booting the server with Hp's Smart Start CD and using the ACU  (Array Configuration Utility) - a GUI array configuration tool. 

I had a 3 physical drive * 72 gb array and added two more 72GB  SCSI drives. One for more capacity one for a HOT SPARE.  (We did have a drive failure in year one which hp hot swapped the next day without the users even knowing a drive had failed. This scared me a bit though as I read somewhere that RAID 5 does not like to run with one failed drive for "too long". How long is "too long" who knows? depends on load I would say.  I picked up the drive failure with a routine hpacucli check that I do when ever i am on site. 

the first bad news is that once I had pushed the button in the ACU to do the array expansion  it took approx 10 hours, Thats saturday gone.  One Saturday evening I then started the extension process and left it running over night so dont know how long this took. So how long would it would take with your much much larger drives. I dont know,  hopefully the latest controllers are faster as drive capacities increase. it is something to keep in the back of your mind though.

So at this stage (now Sunday morning)   boot from the Smart Start CD  (SME is not running at this stage)  and ACU reports an array of three physical disks of 216 GB (3 * 72) plus 1 available hot spare - perfect just what I wanted.

Now take out Smart Start CD and boot SME (with a little trepidation I might add). All comes up fine - great! big sigh of relief.

Do a du -h to see my nice new (bigger) disk and dam it hasn't changed.

I did not expect to see the hot spare but I had hoped for 240Gb on the main drive.

So more reading and basically it seems that the extra space is not available until is is formatted by SME or whatever OS you are using.  The Smart Array just gets a physical disk in place which then must be formatted by your OS.  There is some process where you can effectively do a sort of "live format" of the new space on the drive without (in theory)  destroying the existing data. I have been around computers long enough to know playing safe is safer.

Next weekend I did a full backup,  reinstalled SME with the full format and lo and behold there was my extra space.

So the moral is get the biggest drives you can at day one and allow plenty of time if you ever have to add to an array.

Now you wonder can I just add my hot spare and have it available for SME? 
You will of course need to take the sever down boot with the smart start CD and use ACU to add the drive as a hot spare. You cant just bung it in the server because it wont know what you want to do with it add more capacity or add a hot spare.

Adding a hot spare was minutes not hours like adding additional drives for  more capacity so that is good for you

My gut feeling is that the answer is yes because the hp controller handles the raid 5. Also based on watching while a dead disk was pulled from my server by hp and a new one put in which the controller then happily did the job of intergrating it into the array and moving the data on to it and it all worked.  Now that new disk was unformatted when it went into the server so the controller must have done what ever was necessary.  Seems to me adding a hot spare should be a similar process. When a disk fails the controller detects that and begins the job of moving data onto the spare which will take several hours depending on how much data you have in your case a lot.

So it all sounds a bit complicated but I do think the hp hardware controller is better than the software raid.  The main exposure is a drive failing and it will handle all that without the user having to do anything except plug in a new drive.

Can I ask did you get sata or SAS drives and I would be interested in how you go

hope this all helps  regards mike

[misnerspace]

Hi mwaymouth,

Thanks very much for the detailed (and very useful) extra information, it is really appreciated. I'll be following all your tips regarding mitigating the risks of longer than anticipated downtime. I amwondering whether it would be a good idea to set up additional documentation on this site focusing on SME on HP hardware. Both seem to be remarkably reliable and it seems a good marriage between hardware and software. I think HP is one of the market leaders in server hardware and with my business hat on it seems that it would be a good idea to demonstrate via well supported documentation how easy it is to run SME on HP hardware. Just an idea.