Koozali.org: home of the SME Server

Legacy Forums => General Discussion (Legacy) => Topic started by: beakersloco on January 25, 2006, 12:36:38 PM

Title: Migrating accounts?
Post by: beakersloco on January 25, 2006, 12:36:38 PM
What is the easiest way to move my email accounts + whatever current mail from a modified 5.6 to fresh install of 6.0.1 ?
Title: Migrating accounts?
Post by: byte on January 25, 2006, 12:42:50 PM
depends what's modified, but I would do a backup 2 desktop within the server manager
Title: Migrating accounts?
Post by: beakersloco on January 25, 2006, 12:51:38 PM
I have added several contribs and last time I tried to do a backup and install it on a fresh install it did not work because of all the custom stuff
Title: Migrating accounts?
Post by: byte on January 25, 2006, 01:36:01 PM
try removing them then doing the backup and go to the 6.0.1 contribs, that's what I have seen happen before!
Title: Migrating accounts?
Post by: dmajwool on January 25, 2006, 02:33:51 PM
Quote from: "byte"
try removing them then doing the backup and go to the 6.0.1 contribs, that's what I have seen happen before!

I've seen this advice here a few times, but it seems an unsatisfactory solution because it involves breaking the old server before the new server is proven.  If SME 7.0 is a success, potentially hundreds of us will have modified 6.0 installations wanting to take user accounts and emails across to a new box in an upgrade...  Is there an alternative way to replicate user accounts and mail and nothing else?
Title: Migrating accounts?
Post by: dmac on January 25, 2006, 03:43:02 PM
Could you not make a backup of the /home/e-smith/users? directory.  All the mail information and user data would be in those directories.  Then create the same accounts on the new server and restore the files into the users folder.

Seems simple to me, if you only want to backup user data.

Darin MacLachlan
Title: Migrating accounts?
Post by: dmajwool on January 25, 2006, 04:28:36 PM
Quote from: "dmac"
Could you not make a backup of the /home/e-smith/users? directory. All the mail information and user data would be in those directories.  
Agreed , so far...
Quote from: "dmac"
Then create the same accounts on the new server
Isn't this the crux of the question? How is this done "simply" for a server with 100 users across many groups preserving their passwords and permissions?
Title: Migrating accounts?
Post by: dmac on January 25, 2006, 04:43:27 PM
Quote
Isn't this the crux of the question? How is this done "simply" for a server with 100 users across many groups preserving their passwords and permissions?

You could use the lazyadmin tool set.  

Darin
Title: Migrating accounts?
Post by: dmajwool on January 25, 2006, 04:58:46 PM
Quote
You could use the lazyadmin tool set.  
As described in
http://mirror.contribs.org/smeserver/contribs/mblotwijk/HowToGuides/lazy-admin-tools.htm
lazy admin doesn't have a tool for backing up and restoring user and group accounts and permissions.
Title: Migrating accounts?
Post by: beakersloco on January 25, 2006, 09:23:49 PM
I have heard of the lazyadmin tool set but that still requires that you recreate the usernames. I dont have many users to recreate but figured that people that have hundreds of users and modified systems must have an easy way of migrating the users that doesnt involve recreating each individual user
Title: Migrating accounts?
Post by: dmajwool on January 25, 2006, 11:51:42 PM
Quote from: "dmajwool"
Quote from: "byte"
try removing them then doing the backup and go to the 6.0.1 contribs, that's what I have seen happen before!

I've seen this advice here a few times, but it seems an unsatisfactory solution because it involves breaking the old server before the new server is proven.  

I suppose an approach that avoids my objection would be to do a full backup2ws of the old production server and restore that backup onto a fresh install of the original SME version on the new hardware  to create a clone of the production server. Then that clone could have it's additional contribs removed, and then be upgraded to the next SME OS release.

Is there an approved method for stripping a server back to "plain vanilla" or is it just down to remembering all the rpms of all the contribs that have been installed?
Title: Migrating accounts?
Post by: kruhm on January 27, 2006, 04:46:56 AM
if you don't have much info, you should be able to use a regular backup to do most.

i've been trying to find out if rsync will work but it doesn't seem like anyone has tried this lately. or willing to share info. (look at man rsync for more info)

supposedly like this from the new server:

rsync -vP -z --delete -a -e "ssh -c blowfish" <oldserver>:<old directory> <new directory>

Using:
/home/e-smith/
/home/netlogon/
/etc/e-smith/templates-custom/
/etc/e-smith/templates-user-custom/
/etc/group
/etc/gshadow
/etc/passwd
/etc/shadow
/etc/smbpasswd
/etc/ssh/
/opt/
/root/

I'm still a few weeks away from attempting this but I'll post my results if no one does before then.
Title: Migrating accounts?
Post by: beakersloco on January 31, 2006, 09:36:54 AM
Well I was going to try logging in via ftp with each users account name and copy the directory and guess what the directory structure under each user is different in 6 then in 5.6 .

All I need to do is copy the emails out ..changing the passwords is not a big thing.... anyidea what I need to copy so that the users dont lose any old emails.
Title: Migrating accounts?
Post by: beakersloco on January 31, 2006, 09:42:50 AM
FYI Im about to put a new box into "production" and the only hold up is Im trying to find out if I can move the old mail to the new server.
Title: Rsync migrate from 6 to 7
Post by: silasp on February 28, 2006, 12:28:42 PM
[updated 1-Mar-06: with passwordless ssh keys setup details]
[updated 2-Mar-06: SME6 => SME 6 successful]
[updated 16-Mar-06: The way that worked]

This will upgrade SME 6 => SME 7, while migrating from an old server to a new server. It may also work for other SME versions, and can be used for offsite backups with a little modification.

Warning. This worked for me, but it may not work for you.

You will need:
- Old server (SME 6) with data
- New server - must have enough HD space to hold old server data.

Install SME 7 on the new server.
Configure it in "server-only" mode, set the IP to something outside the DHCP range, turn DHCP off on the new server.
I gave new server a unique name, and new domain the same value as old - but it really shouldn't matter

You may need to configure the new server to allow remote SSH access at this point - from the "Remote access" option in the server manager.

Log in as root to the local (new) server, enter
cd /root/.ssh
ssh-keygen -t rsa
(choose a randomish name eg "keyname" and press enter twice when prompted for a passphrase)

chmod 600 keyname*
scp keyname.pub [ip of remote server]:/root/.ssh/
(enter root password)

- Now connect to the Old server (replace 192.168.1.1 with actual ip)
ssh 192.168.1.1
(log on as root)
cd /root/.ssh

if [ ! -f authorized_keys ]; then touch authorized_keys ; chmod 600 authorized_keys ; fi
(above "if" command should all be on one line)
cat keyname.pub >> authorized_keys
rm -f keyname.pub
exit

- Now you're back in local (new) server shell

Create the following script, and replace 192.168.1.1 with the IP of your old (remote) server.
Note that each rsync command should be on a single line.

pico /migrate.sh
Code: [Select]

# Make sure you check
# /usr/lib/perl5/site_perl/esmith/Backup.pm
# for the correct backup file list for your SME version
# it begins with "sub restore_list" (near line 70)

# Replace this IP with your remote host IP
RMH=192.168.1.1

cd /
# All these are directories - note trailing slashes
rsync -vPa -e ssh $RMH:/home/e-smith/ /home/e-smith/
rsync -vPa -e ssh $RMH:/etc/e-smith/templates-custom/ /etc/e-smith/templates-custom/
rsync -vPa -e ssh $RMH:/etc/e-smith/templates-user-custom/ /etc/e-smith/templates-user-custom/

# Group, gshadow, passwd and shadow are in a different format, so
# don't overwrite these

mkdir /etc/pwold

# All these are files - note no trailing slashes
rsync -vPa -e ssh $RMH:/etc/group /etc/pwold/group
rsync -vPa -e ssh $RMH:/etc/gshadow /etc/pwold/gshadow
rsync -vPa -e ssh $RMH:/etc/passwd /etc/pwold/passwd
rsync -vPa -e ssh $RMH:/etc/shadow /etc/pwold/shadow
rsync -vPa -e ssh $RMH:/etc/samba/smbpasswd /etc/samba/smbpasswd
rsync -vPa -e ssh $RMH:/etc/samba/secrets.tdb /etc/samba/secrets.tdb
rsync -vPa -e ssh $RMH:/etc/smbpasswd /etc/smbpasswd
rsync -vPa -e ssh $RMH:/etc/sudoers /etc/sudoers

# More directories - trailing slashes
rsync -vPa -e ssh $RMH:/etc/ssh/ /etc/ssh/
rsync -vPa -e ssh $RMH:/root/ /root/


# Assemble and reboot
/sbin/e-smith/signal-event post-upgrade
echo "Now you need to execute /sbin/e-smith/signal-event reboot"
echo "Finished"


chmod 500 migrate.sh
./migrate.sh

Now walk away for a few hours. It takes about 1 hour per 10 Gig of data.
Backup whatever mysql databases you may be using (except for horde).

/sbin/e-smith/signal-event reboot

Plug in the new server where the old one was, switch off the old server.

Login to admin on new server and reconfigure it. You will probably have to change the network drivers. The other settings will be the same.

Test that internet still works.

Now the annoying bit. You need to "merge" the contents of the "old" and "new" group, gshadow, passwd and shadow files, as follows.

- Backup the existing (new) versions:
cd /etc
mkdir pwnew
cp group pwnew
cp gshadow pwnew
cp passwd pwnew
cp shadow pwnew

mkdir pwmerged

Now you have an old and a new version of these four files. Here's what I did to merge them, maybe some Linux god can create a script to do this:

For the group, gshadow, passwd and shadow files:
[1] Copy the entire contents of the old version of the file in question to something like M$ word. (Open Office will probably do the job, maybe even Vi?). Just copy and paste from the SSH client.
[2] Select all of the old version in word, and change the text colour to red
[3] Copy the entire contents of the new version of the file in question to the same word doc (or whatever).
[4] Sort the file into alphabetical order by paragraphs (ascending)
[5] Now go through each user / group in the file. Where there are double-ups, you generally need to delete the RED line - unless it is stacked up with users separated by commas at the end, in which case you just need to change the formatting at the start to the same as the BLACK line and delete the BLACK line.
[6] Go back to your SSH client, pico -w pwmerged/group
(or /gshadow, or /passwd, or /shadow) and paste in the contents of the word doc. Save it.
[7] Repeat steps 1 to 4 until you have merged all four files

Now
cd /etc/pwmerged
cp * /etc/

Pray that it worked
signal-event post-upgrade; signal-event reboot

System reboots.

(optional)
Install contribs.
- sme7admin
http://sme.firewall-services.com/
- Phpmyadmin
http://mirror.contribs.org/smeserver/contribs//dmay/smeserver/7.x/smeserver-phpmyadmin-2.6.4-pl4.dmay.noarch.rpm
- Traffic shaping script
http://www.hanscees.com/sme7/wondershaperbeefedup.html
and any others you might like.

Now try logging on from the windoze machines on the network. No luck? In that case, you will have to log on as administrator to the local machine of each windoze box on your network and
- rejoin them to the domain. (join them to a workgroup "nothing", then join them to the domain. restart)

then if not using roaming profiles, on each windoze box:

- logon as user, log off
- log on as admin:
- copy old profile to new profile (look in c:\documents and settings and try xcopy oldprofiledir newprofiledir /S /C /I /H /R /K /X /Y from a command prompt)
- control panel > users and pws > advanced > advanced > groups (double-click) > administrators (right click) > add to group. Remove all members except Administrator and BIOCON.LAN\Domain Admins. Click Add, change look in to your domain, scroll down, click Domain Users, click add. Click ok x 2. Close windows and log off.
- Logon as user again, check that everything works, especially email and printers.

Cheers,
Silas.
Title: RSYNC SME 6 => SME 6 (new install) => SME 7 (upgrade)
Post by: silasp on March 02, 2006, 03:52:41 AM
Well, it almost worked!

SME 6 (modified) => SME 6 (clean) worked great!

SME 7 upgrade install went ok, but on reboot I'm getting "/etc/httpd/modules/libperl.so missing" errors with my version of /etc/httpd/conf/httpd.conf

Error keeps repeating every second.
Fixed it temporarily by
1: alt-[f2]
2: logon as root
3: pico /etc/httpd/conf/httpd.conf
4: press [f1], make a note of the line number causing that error
([ctrl]-w, [ctrl]-t, type the line number)
5: add # to start of that line
6: Repeat steps 4 and 5 until error messages stop.

Updating / Reinstalling PHP + Apache on the new box will probably solve this problem.

That looks ok now. I have run a few basic tests - all the user accounts and emails came across ok.  Horde works ok, have not had a chance to test with the windows machines on the domain - to see if they still log in, etc.

When I do this for real, I will run the SME 6 "rollback" script
http://no.longer.valid/phpwiki/index.php/SME6.0.1Contribs-RemovalScript

Something like:
fresh install SME 6 on test box.
real server (modded sme6) => rsync => test box (sme6 new install)
run rollback script on real server
upgrade install SME 7 on real server

(if it works, great, otherwise...)
install SME 6 on real server
test box => Rsync => real server
run rollback script on real server
upgrade install SME 7 on real server

Cheers,
Silas
Title: Actually, the best way is:
Post by: silasp on March 14, 2006, 04:45:27 AM
Hello again.

I have since found that the best way to move from SME 6 => 7 is to backup data,
do a CLEAN install of 7,
backup passwd, group, gshadow and shadow files on new server,
restore data,
fix passwd (etc) files.

Check here
http://forums.contribs.org/index.php?topic=31159.msg130622#msg130622 for details.

Cheers,
Silas.