Topic: Adding a 2nd RAID array to SME -- [2 Different Solutions]

[Mike]

I've used SME for about 2 years now and found it to be a good and reliable work horse.

For the last 6 months I've been running on 7.0rc2 and my current use is mostly as a networked file server within a Windows environment.

Over time I've evolved a setup which really suits the way I work ...

Basically I have three 300Gb drives, one is permanently fixed in the machine (hda) and the other two are mounted in caddies. At any time one is in the machine and acts as hdc, the other is stored in a safe place.

Every 2-3 days I power down the machine, swap the caddies, power-up and rebuild the raid arrays using PuTTY and mdadm from one of the other machines. It's a great and painless way of getting a RAID plus an up to date backup!

It works like a dream for what we want/need.

Now I'd like to do two things ...

The first is to upgrade to the 7.0 release version -- that's no problem, I can handle that ...

The second is to increase the amount of storage ... the present array gives me effectively 300Gb. Because of a machine reorganization I have three spare identical 160Gb drives and I'd like to use these to create another array within the server. (So that I get the same backup/reliability features with an extra 160Gb of storage -- of course I'd have four caddies in total then.)

I've spent the afternoon browsing around the forums and looking through the howto's but I've not come across anything that tells me whether or not I can do what I'm trying to do. If anything the suggestion is that I can't easily do what I want.

I know the performance/reliability issues with putting master+slaves onto IDE controllers -- if I can get this configuration up and running then I'll add extra controllers into the machine via the PCI bus to overcome those.

What I really need to know is ... Will the SME installation allow me to put 2*300 and 2*160 drives into the system and configure them as an array of 300's and a second array of 160's?

If I can't do it at installation time, can I do it afterwards? (and as a sub-question, can I do it easily?)

As I say, I've spent some time looking for the answers, but if I've missed the ideal/obvious write up of the solution I'd be very grateful for a pointer to the posting or the page ...

Many thanks,

Mike.

[CharlieBrady]

What I really need to know is ... Will the SME installation allow me to put 2*300 and 2*160 drives into the system and configure them as an array of 300's and a second array of 160's?

No. Standard installation allows you to have any number of identical drives, which will be automatically built into a single RAID1, RAID5 or RAID6 set.

[Mike]

Charlie,

Thanks for the reply ... that's what my 'subconscious' assimilation of all that I read this afternoon suggested.

I guess the standard installation will either stop when it sees a mixture of drives or try to make some kind of minimal (but single) array out of them.

Ah, well, that's the easy option out of the window ... it looks like I'll have to 'knife and fork it' ...

Now for the second part then, can I manually configure a second array of the 160's to sit alongside the first array of the 300's --- effectively creating md3 to use hdb+hdd? (SME seems to use md1/md2 for its swap and the main array).

And if I can configure such a beast, any pointers on how to go about it?

Mike.

[kruhm]

The by the book answer is: no.

This is for various reasons to save you headaches along with everyone else.

It sounds like you know what you're talking about. So let's think through this.

The system will recognize the drives and you should be able to configure what you want with LVM. Look at the man pages for LVM and MDADM. Or spend 2hrs a day for a week studying.

You can even get the system to recognize the newly created MD upon reboot if you tinker a little. The problem is keeping it that way through upgrades and patches. The devs aren't going to be able to support and take in consideration every setup. They are just going to focus and what's listed above -identical drives.

You'll probably run into problems during the updates as the packages overwrite files and either you are going to have to fix it every time or ask for help. To prevent this and to make everyone's life easier the general answer is to not delve into this too much or at all.

[Mike]

As SME is a great tool for making use of old systems and old disks as a general utility file store I can't help thinking that someone out there must have done this (or attempted it) before. After all, older salvage disks tend to be smaller and the natural thing is to put several together to make a decent storage size -- so making a RAID array with two sets of disks in it seems a good way of recycling those drives coming out of upgraded machines.

I can understand that the main development effort doesn't want this type of customization on their already full plate and that with anything along these lines I'd be more or less 'on my own' and have to be self-supporting.

Thanks for the tips about lvm/mdadm ... I already have some familiarity with mdadm as I use that to monitor the existing array and to rebuild it each time that I extract my backup disk ...

Currently my thoughts/plan are to

Backup everything ...

Remove the existing CD-Rom (I think its unplugged anyway)

Install a couple of the 160 drives as slaves on IDE0/1 (I know about the performance hit and will solve that later with additional controller cards)

The BIOS should see the drives ...

Partition and format the drives ...

but not link them into the directory structure anywhere (I seem to remember there's some good HowTo type info on adding non-RAID disks available.)

Then use mdadm to make the second array (actually md3) out of the two disks.

Finally link the new array into the directory structure at a convenient point
[/list]
As I say I'm sure someone must have tried this before ... I'd certainly be keen to know how they got on with it.

Mike

[Mike]

I've been playing around with a sandbox system seeing what I can do/achieve.

Firstly ... adding in extra IDE controller cards is no problem ... just gives me access to devices hde-hdg as I expected.

Secondly ... for some reason I can't get a 3-disk RAID5 running properly -- but don't worry about that, that's not my main aim in all this, just something I did for curiosity (and I don't want it to kill the cat).

Thirdly ... if I build a system with a 2-disk RAID1 ... mdadm allows me to add a third disk to it to make it a 3-disk RAID1.

What about this for a plan of action ... ???

Build an SME 7.0 system with NO raid, just a 300 + 160 Mb disk drive in it ... then create RAID1 arrays based on the 300 and the 160 by partitioning/formatting drives, creating the arrays and adding the drives to them one at a time?

Will this work or am I just letting myself in for a whole lot of trouble?

[raem]

Mike

> Will this work or am I just letting myself in for a whole lot of trouble?

I think kruhm already answered that.

[andy_wismer]

Hi Mike

I've setup quite a few SME 6 and 7 systems for clients all over Switzerland.

I have a few clients where I use the following setup:

HP ML110 Server (Low-End, but nice, small & quiet!)
2 * 80 GB SATA as System/Data Disks (Some Clients use 2*250 GB SATA), setup by SME7 as standard SME RAID1.
1 250 (or larger) Disk as Online-Save with 5 or 7 Generations of ALL data, incl. /etc, /home, /var and DB Dumps if needed.

Remarks:
System setup standard, with two same disks, third disk NOT Connected!
Third Disk linked in after SME Setup.
As this Disk is only local backup (already redundant), no RAID needed.
Most Clients have Offsite-Backup of this entire volume offsite, e.g. at home, using rsync.


Other Clients use a form of HW Raid.

5 Disks, Client wants system as Raid 1, Data as Raid 5.

Remarks:
Using Raid-BIOS setup two separate RAID Arrays, one with two disk mirrored, the other using Raid 5.
Physically unhook the RAID5 disks and setup SME. As SME will only "see" one disk, Softraid isn't really used. (Yes, it does create a RAID1 comprising ONE disk...)
When done, reboot and hook up the Raid5 disks.
Link them in the file-system as needed.

I try to leave the system area of SME intact so to avoid headaches during updates and/or disaster-recovery.

All of the servers are running like rock. The "fattest" was a recent client with a new ASUS 3.2 GHz Server with a 3Raid HW Controller and almost 1 TB of usable file-space. That server went alongside their existing 2.8 GHz Windows 2003 SBS server, the SME as member-server. The client was a first time Linux user (25 employees & PCs, a lot of CAD and huge geographic files). All are VERY happy with the speed the SME servers up the files.

My 2 cents...

Andy Wismer
ANWI Consulting
www.anwi.net

[Mike]

Andy,

Thanks for the very encouraging reply ... I've been doing some extensive playing with my sandbox system and I can now do everything that I want/need to do ...

There's only one problem left and that's got me really stumped !!

I can set everything up, partitition, format, create the array, mount it, sym-link it in ... but when the system reboots it always fails as the physical device size of the array device is different to what the super block says.

If I scrub the drives, partition them, reformat them and then create an array from them -- then run fsck on the array device I get the following ...

[root@aboo ~]# fsck /dev/md3
fsck 1.35 (28-Feb-2004)
e2fsck 1.35 (28-Feb-2004)
The filesystem size (according to the superblock) is 40019915 blocks
The physical size of the device is 40019888 blocks
Either the superblock or the partition table is likely to be corrupt!
Abort<y>?

If I fsck the individual devices (hde1 & hdg1) then they are ok ...

I'm really stuck with this one -- am I doing something dumb ? Has anyone got any ideas what might be causing this and how to fix it ?

(BTW: I can't seem to get fsck to fix it ... )

Mike.

Stop Press: I may have solved it ... looks like you have to format the RAID array device after you have created it ... rather than using the previously left-over formated disk drives by chance. More on the subject after I've had a chance to test it thoroughly.

Yes that is it ... format the RAID array rather than the individual drives ... basically the sequence is ... scrub - partition - create raid - format raid device - mount and link into existing file structure at some convenient place.

[Mike]

Since the last post I've gone on to solve the overall requirement/problem in a different way.

I've RAIDed the new disks and added them as a physical volume into the LVM main group, then extended the existing file system to utilise the new space.

This is a different solution to what went above in that there is now no singularly identifiable disk with a specific part of the file system on it. In otherwords, whereas before I had one disk with SME, iBays etc on and a second disk with my network backups. Now I have a pair of disks that are both required to produce a viable system.

Whether this is a better or worse solution is a good question ... I gain flexibility of storage management, but I may loose some disaster recovery type integrity. I know I can recover from the two disks combined ... but I won't be able to take just one of the disks to another box and recover it seperately.

I'd be interested in peoples thoughts about that last issue ...