Topic: Server Migration (to new machine)

[elSpike]

Hi there e-smithians.

I have just completed a migration of a 5.6u4 server from an old p3 850 to a new Dell 2600.

Due to the massive amount of data (about 40Gb) in our home and ibays the traditional backups and backup2ws were either a hassle to do or plain impossible.

So I managed to sortof modify the backup2ws technique to allow me to move in the minimum of time.

I installed 5.6 and patched it to U4 (and added in all the contribs etc.)

Then basically all I did was use the command:
rsync -vP -z --delete -a -e "ssh -c blowfish" :/


for each of the following: (as stolen from the DR backup in backup2ws)

/home/e-smith,
/home/netlogon,
/etc/e-smith/templates-custom,
/etc/e-smith/templates-user-custom,
/etc/group,
/etc/gshadow,
/etc/passwd,
/etc/shadow,
/etc/smbpasswd,
/etc/ssh,
/opt,
/root

once they were all rsynced across to the freshly installed and patched dell 2600, I just triggered a signal-even post-upgrade and rebooted.

voila, new server up and running.

the only things i needed to change were:
re-add me and my techs to the sudoers file.
clear out the old /root/.ssh/known-hosts keys.

I will write this up as a how-to soon and post it here once done. Perhaps even write a quick perl script to do it for you.

cheers
elSpike out.

[elSpike]

that command should have read:

rsync -vP -z --delete -a -e "ssh -c blowfish" :/

[elSpike]

add a space and a < new dir > at the end of it.. stupid forum!

[Ray Mitchell]

Dear elSpike

I'm sure people will appreciate that HOWTO, when they need it. Please do finish it and post here or to e-smith.org HOWTO's or to contribs.org.
Your migration task will become a common one for users as hardware is upgraded etc.

Regards
Ray

[Walter Padgett]

Good Evening,

The information looks great. There was a posting not to long ago about domain controllers that would mirror each other. Sort of as a PDC and BDC relationship. Would this option that you have presented be somewhat of a solution? I'm guessing that it could be scheduled on a fairly regular basis? I'm interested in how much time/bandwidth you consumed in what you did.

Am terribly interested in the how-to and/or perl script that is coming.

Later

Walter "Wally" Padgett

[elSpike]

Hey Wally,

You would need to tweak what information was transfered as in this case I essentially cloned the machine, which meant that when I initiated the post upgrade event and rebooted it took on every characteristic of the old server (ie ip address, domain name etc.) but in theory it could work as rsync is incrimental.

As to time and bandwidth....

hmmm well after my first go at it where the /home/e-smith files all ended up in /home/e-smith/e-smith (note to self read rsync man page to find out about trailing /'s) it took about 4 hours to transfer the data over a 100Mbit connection (switched network of about 200 computers). which in terms of a complete bare metal restore from backup wasnt too bad.

I will press forward with the howto then.

Reguards elSpike - WA Australia!

[Karl]

Can't wait to see/read the how to in more depths...... I await your return....

[Graeme Fleming]

Yo Spike

Fellow WA person

Any thought about making this a panel in server manager as well.  If the copy goes ahead based on the set schedule in the panel then all that would need to be done to bring the duplicate up in case of primary failure would be to tick a checkbox in the panel on the backup box and say go!

Another thought is: What happens when the primary is back up again?  How do we demote the backup?  Perhaps a demote button on the panel can change the primary IP back to its original, turn off DHCP and then reboot, this would tehn allow it to resume its backup duties (especially as the backup box's IP would be in the hostnames panel in the primary and the username passwords would match (for now).  Perhaps authenticate on boh boxes with a static account/password combo.

If I can help let me know (have a few test boxes to play with if req).

[Walter Padgett]

Good Morning again,

Question regarding this issue. I would like to backup my current server onto another machine so I can reinstall 5.6u4 on the original server. Will this backup all of the userlists as well as their email? Also, if I'm currently having trouble with some contribs and vpn's, will I continue to have those if I use this backup method?

Later

Wally

[elSpike]

Firstly, Graeme:

Yo Spike

>Fellow WA person
yup and PLUG member too

I guess just having a panel prompt that says - make this primary would be doable. all you need to do is canabalise the "advanced restore thingy" from backup2ws panel and rename it.

once the data is over all that is needed is for the following commands.

/sbin/e-smith/signal-event post-upgrade, and
/sbin/e-smith/signal-event reboot.

The only issue i can see is if the backup server accidently looses power and reboots with all the new config files in the directories... which opens up a whole kettle of worms with two machines trying to be the same IP address. the other issue i found was i needed to go into the admin console and re-impliment the networking drivers as i had two different network brands in the different servers. but with a little tweaking of what is coppied it can be done... I wish I had more time to play.

>Another thought is: What happens when the primary is back up again? How do we demote the backup?

this ties in with the above. basically the backup machine/s would need two configs.. one for when it was "backup" and one for when it is doing the Primary role.

I guess we could use some sort of sim-linking to different configs to fix this.


Next Walter,

>Will this backup all of the userlists as well as their email? Also, if I'm currently having trouble with some contribs and vpn's, will I continue to have those if I use this backup method?

this will do exactly what the backup2ws disaster recovery default backup will do. it restores all users/passwords/ibays etc so yes it will do all the email etc.

as to the next part - im not sure if I understand. are you asking if this copys contribs etc over? - No it doesnt - you need to install all the base system and contribs before rsyncing across. The other way i could read that question is if this backup will fix all your issues with contribs/vpns.... I dont think so.

HTH. I will have a 1st draft on the How-to by monday i think. will post here. Graeme, will mail you off list to chat about maybe making a user-panel contrib.

[elSpike]

Version 0.01 of the HOW-TO is located:

http://www.tehvand.com/e-smith/rsyncmigrate.html

please offer sugestions and coments as you feel lead. I am a sysadmin not a document writer.

elSpike out.

[Kelvin]

Hi elSpike,

I was wondering if you could narrow the scope of what is backed up ? All I want is the user / group / pseudonym / passwords and absolutely nothing else. This would be a great time saver for those who only need to either migrate users (or replicate them from a master server). Ibays, not interested, as I can backup / restore them separately efficiently enough if I need to. Configurations, not interested as well, I prefer to configure the servers individually.

One example just as I'm typing this.

I need to urgently replace one SME with another (same versions on both servers but fairly different hardware).

I moved the ibays out of the way then did a backup to desktop.
Then restored the back to the new PC.
Rebooted the new server and ended up with a kernel panic because the updated storage device driver for the new server will not load (why it changed anything from the desktop backup I don't know), hence cannot find root. Sure we could work around this problem. However, if I only could backup and restore only accounts and passwords, I could have avoided this hassle to begin with.

Kelvin

[elSpike]

Kelvin.

It wouldnt be an issue to narrow (or extend for that matter) the rsyncing that occurs. Im sure someone could tell us the files that are required for a user/psudonym only backup/migration. I havent looked into it all that deep yet.

I assume copying over the /etc/passwd, /etc/group, /etc/gshadow, /etc/shadow and the applicable mysql data (how e-smith logs the usernames etc.) would do the trick. Im not sure what mysql-data would be required (if at all as the signal-event post-upgrade may rebuild the tables)

Perhaps a question to the e-smith maintainers on this matter?

cheers
elSpike

[Kelvin]

Hi elSpike,

>Perhaps a question to the e-smith maintainers on this matter?

Believe me, I've posted a number of times before. The fact that I'm asking you says it all.

Kelvin

[Graeme Fleming]

Could be cause to move this discussion to the devinfo list to iron out the whys and the wherefores.  Once this has been done then we can post back here to tell everyone whats going on.

[Ray Mitchell]

There were posts previously in these forums that also touched on this issue.

Wasn't the problem something like that the password files were encrypted for the particular setup/configuration and if you move just the password files it does not work on the new server whereas if you move the whole server setup it's OK . ??

Kelvin, you would probably remember the particular post that covered this.

Regards
Ray

[Kelvin]

Hi Ray,

Yes, I do remember the post but it was not confirmed by the powers that be that this was indeed the case. If it were really encrypted by a key generated by each setup, then the key itself is likely able to be backed up and restored to a new server as well, if we knew what to backup and what relies on that key.

This is the same with NT / W2K servers. M$ never did properly address the issue and so 3rd party software vendors did.

Kelvin

[elSpike]

Hi Guys,

Have been playing a bit with the information that is rsynced across. I am trying to elimenate the "data" and only copy across the acounts info etc.

Not knowing how the server actually records account info (ie i know it isnt just a simple case of passwd,shadow,group etc.) it is proving interesting.

So far the best I have been able to do is copy everything bar the data in ibays and users except you need to keep all the directories in these or the accounts are useless.

will keep you posted.

elSpike out.

[Dan Brown]

You'll want to copy /home/e-smith/accounts, and probably /home/e-smith/configuration (in addition to passwd, smbpasswd, etc.)