Topic: How to syncronise 2 sme servers

[torsten]

I would like to have a 2nd sme server which would kick in if the primary server goes down but of course the main thing is that they both have identical data since the last sync.
I know there is rsync but I couldn't get that going and still scratching my head.

Anyone done this before?

Thanks T

[arthurhanlon]

Hi there,

Would it not be more feasible to look into a mirrored RAID array? Although I have not actually tried it myself, I hear that the software RAID functionality is very good and from what I gather, the hard disk is the most likely thing to "pop" in your server so a mirrored array would probably suffice and save from having two physical boxes.

Just a thought,

Arthur

[torsten]

Thanks Arthur,

I was thinking along the same idea...
but as a contingency I thought having a server ready to go would be good.
I had lately a lot of bad luck with my power supply in the servers, not that it is a big deal but takes time to get replaced

T

[arthurhanlon]

I can appreciate that, my PSU went not so long ago but luckily I had one to hand to replace it with. At the end of the day, I think it all boils down to how critical it is to have your server running.

If you want to go down that path, I would install removable caddies in the servers that way if hardware should fail in one you can just whip out the disk and slot it into the emergency box. No need for the complexity of syncing two physical boxes together. Not sure how well that would work for hardware differences though, maybe someone else can comment?

Again, just a thought,

Arthur  

[torsten]

I just found this site
http://www.tech-geeks.org/contrib/loveless/SMEServer/howtos/rsync_backup/SME_BackupServer.html
That might be the solution to the problem...

[Morgo]

Hey Guys,

Mirroring is great for redundancy as you hards will fail, its only a matter of when? Unfortunately raid doesnt help if some low life steals your server (that is usually also a matter of when!). Remember the 3 golden rules of IT - BACKUP, BACKUP AND BACKUP. I would love to here any comments on rsync.

Paul  

[torsten]

sure thing Paul you cannot beat backups but that either requires tapes, or like Arthur suggested additional hdd or another syncronised server.
The link to tech-geeks has a good foundation and I will try to modify that to my environment.

Get a primary sme serverwhich does my day to day web serving and emailing
Then I will have a secondary server running and might get them to sync evry hour or so
Once one dies the alarm bells go off and I can quickly action it
Wow that sounds like a plan

T

[chris burnat]

Torsten, I agree, two boxes are better than one anytime.  Boxes are cheap nowadays, data cannot be replaced easily.  If you do not have time to implement an in-house solution to this problem, you could use dungog rsync and dungog backup. I have a couple of mission critical clients with two identical servers.  (i) configuration files and users are backup from one box to another in tgz format every day.  You may also wish to backup a few essential ibays.  This allow rebuilding a box in a short time.  (ii) Rsync is used to "mirror" ibays from one server (Main) to the other (Standby). Alternatively, essential data such as financial etc can be backed up using Rsync over ssh to an offsite box.  To add some security to this off-site backup scheme, ssh port 22 is changed to a higher port, and password for the rsync operator is made very rugged. Hope it helps. chris.

PS: disk are cheap as well, so do a raid1 on each server.

[arne]

I did some experiments on sme backups a for a while ago.

Først I tried to make a routine for automated data backup with a serial number/day system so I just filled a hd no 2 with a series of bacupfiles.

Well it worked, but it filled up with a lot of datas, but my thought around it was this that the bacupdatas were not realy safe, as long as on the same PC.

Then I started to think about a inexpencive storage medium, and I found out that a USB harddisk should be one of the bether choices (??).

Then I started to develop a solution based on the use of the principle of making bacup using a number of cheap USB harddisks for full data backups.

The tasks were automated in such a way that you just had to plug in the usb harddisk for instance in the evening and pick up the full backup in the morning.

This solution were improved until I could not imagine how to improve it more, and then the whole project were packed down, because I really do not use such a backup capacity for my homeserver. The whole project were made mostly for "fun" and the exitement to see if it could work.

I don't know if usb harrdiks is considered to be a "safe enough" medium for backup, but it seems to work rather good. By the way for making it I hacked into the backupfunctions of the sme server to make bacups of the identical same files including dumps from the databases.

What about starting up such a project "backup to the usb harddisk" and then this time with someone with the knowledge of making a server-manager front end ?! (in my project it was just a collection of shell scripts.)

Don't know why USB harddisks are not used a lot as becup media. (Well you could pull out the cable or something but this will normally not happend.)

Arne.

[arne]

By the way, one thing I also tried was to syncronize to sme servers using rsync via nework. I could not find out about how to use rsync over the net, so thats the reason that I used rsync to make a data copy to a USB disk.

If someone know some details about how to ser up a rsync server and a rsync client and how to rsync via the network, the principles from the USB backup disc should be quite transferable to a principle of a complete syncronization of to servers.

I had to give up my rsync via net project, but I think it could have worked rather nice, if it had just worked at all. Anybody who knows how to do that ?

Arne.

[Brenno]

I'm with arne on this one... using rsync to backup remote servers would be great.  I can get it working fine on a local network, but never from a remote network.  I'm trying to backup a few Windows servers using openssh and sync2nas.  I'm close, but stumped. It just doesn't work and nothing ever shows up in the SME logs for clues.

Any ideas?

[arne]

At the moment I must admit that I do not understand the info contained in this link, but I believe it might have some interesing content ..

http://www.rackaid.com/resources/tips/rsync-ssh.cfm

[arne]

The rason why i could not find out about using rsync the last time was because I could not find out about using the rsync server.

What is "the rsync program" is a client or is it a server, or is it both ? How do you eventually start the rsync deamon ?

If you combine the use of a rsync client with some other deamon, lets say a ssh deamon, could you then logon using a "rsync client" from the backupstation to the ssh deamon at the server that should be backed up, using the rsync to "pull out the file copies" from the server that should be backed up to the "backupserver" ?

For me this sounds a little bit reasonable because when you use the winscp program on Windows and against a ssh deamon you are doing some manual file management that is a little bit equal ??!!

The man page for rsync:
http://samba.anu.edu.au/ftp/rsync/rsync.html

Is is anyboudy of you who know if there really exist something like a rsync deamon, and if that is the same or something different than the rsync vlient program. I would gues that "using the ssh deamon method" could work, but it would be interesing to know and understand "the full story" ..

Arne.

[arne]

well this gives me at last a prompt for a password .. have never come as long as that before ..

 rsync --archive --rsh=/usr/bin/ssh --verbose root@domene1.com:./arne ./arne

Arne

[arne]

This actually worked, syncronising the folder /arne from a remoote sme 6.0.1 over to a loval Linux Centos server.

I think the way it works is by using the ssh deamon on the sme server for logon, transport via ssh and then the rsync client on the local Centos server (that could had been a sme server as well, but it is a Centos test server just now.)

 rsync --archive --rsh=/usr/bin/ssh --verbose root@domene1.com:/arne /arne

OK, I think that this can work and the rest of the problem is to find out wich files to syncronize and how to dump and restore database files etc.

I think the last time I did this (with a usb harddisk) I hacked into the sme bacup routines. Dont remember where, but I think it was somewhere a clear text file where you could read out all info about the standard sme backup.

I have just tested it a little bit, but just now and initially it looks like it should be possible to syncronize any servers anywhere via Internet.

Arne.