Topic: Should I upgrade from V8?

[david000]

I'm running SME v8 and all's well.

I'm setting up a spare server just in case the one we are using fails and wondered if it's time to move to v9. Any thoughts ?

We're running Roundcube web mail but it would be cool to move to a up to date version of horde.  Other than that I expect our set-up is pretty vanilla. One possibility is to run SME as a domain controller - we're a small business and need to get a handle on user accounts.

On the subject of building a backup server - is it possible to configure two SME boxes and have one mirror the other ?

Thanks in advance.

[guest22]

SME Server 9.x is the latest version and on par with CentOS 6.x. New features and development will target SME Server 9.x 64 bit only. SME Server 8.x is in 'idle' or 'maintenance' mode only, read security fixes until EOL.

In short, all you deploy as new or replacement should be SME Server 9.x

[mercyh]

On the subject of building a backup server - is it possible to configure two SME boxes and have one mirror the other ?

Yes, but maybe not exactly how you are thinking.

I have been running Affa since SME version 7.x and it works great...

This will not be a live mirror but you can set the backup\sync schedule to multiple times a day if needed....
With the "Rise" command, the backup server is cloned as the production server. This process takes less than 30min in most situations.

[david000]

Thank you.

RequestedDeletion, V9 it is then.   The only slight glitch is that I don't think I'm running it on a 64bit box as yet.   I assume that's not an issue - I'm downloading the  i386 iso as I type.

mercyh,  That sounds ideal. The idea is to be able to get a 'spare' box online with the minimum of trouble if need be. I'll check Affa  

[janet]

david000

sme v9 64 bit supports Software Collections which allows multiple versions of the same packages (eg php, mysql) to be installed & utilised by other packages requiring a certain version of dependency rpms.
see

sme v9 32bit only has very limited support for Software Collections mainly from very few external non Red Hat repositories.

If you do not need to run bleeding edge versions of packages then 32 bit should be OK.
It has been said that there is very little speed difference between equivalent machines running 32 or 64 bit server versions.
The need for 64 bit OS will probably depend on how demanding your server requirements are, eg for server side applications etc.
It seems new hardware is mostly 64 bit, so eventually we will all have to go the 64 bit path for sme server OS.

"Rising" an Affa backup server takes about 20 minutes, compared to many hours if restoring from a USB (or similar etc) backup, so speed is Affa's advantage.
IIRC you still need to reinstall contribs so there could be an hour to a few hours of work to get the risen server looking "identical".
Then as well you still need to resolve & fix whatever problem occurred with the main server & restore from your Affa backups, which can still take many hours, so there can still be many hours work required "behind the scenes".

You could in theory just have a basically configured sme standby server (say with users only & some standard data), which you could have running at the flick of a switch (or the plugin of a cable), & that then gives you time to rebuild your main server behind the scenes (eg by fixing the hardware problem, installing the clean OS & restoring from backup). You then need to add in any changes made to the standby server eg incoming mail messages etc.

AFAIK no one has really yet developed a true hot swap standby server "contrib or howto", although the technical bits & pieces seem to exist.

You could also achieve a similar end result by having a highly reliable main server, say with various spare parts, motherboard, power supply, RAM etc, & use a 2 or 3 disk RAID1 (disks in removable caddies or drive slots) with a few spare disks you swap daily & rebuild the array each time.
In the event of a failure you can use one of the removed hard drives to get a degraded array up & running in a few minutes.
The data would be as good (ie current) as at the time of the most recent disk swap. More technically involved, but less work to do to rebuild the server, as all contribs etc are already installed (on the RAID1 drive).

[david000]

Thanks Janet.

We're a small office with just a few account so I would guess not very demanding. I'm downloading the x64 version at the mo and will see if it will take. If it does not i don't think it will be a problem. I doubt that at the moment we need bleeding edge, after all, things are fine on the v8 server which is x32.

My idea is to set up a standby server (exactly the same hardware as the main one) which can be switched in when needed. If I understand correctly, I can create a duplicate SME server complete with the same contribs, and use Affa to schedule back ups from the main to the duplicate. Does it follow that after 'Rising' the new one would match the original, complete with contribs ?

Presumably them, the 'backup' server becomes the new main server and then the problem is fixed the 'old' can then be the reserve ?  This would save having to try and merge or sync the difference between the two email sets.


Another quick question - is there a quick way to check what contribs I have on the existing v8 server ?

[DanB35]

On the 64-bit question, what's your hardware?  If it's at all modern, it should be fine. If you're using an old Pentium II, maybe not.

[janet]

david000

You cannot install the 64 bit OS on 32 bit hardware.
If you are reusing 32 bit hardware, then you need to install from the 32 bit OS media.

Is the main sme server a Domain Controller (DC)., if so it will have machine accounts for the workstation login (which are unique), so you cannot just create a duplicate server, you will have to backup & restore from the main server to the backup server.

Affa will backup the standard sme server backup folders.
It has been a while since I did a Affa rise, but  I think you still need to reinstall contribs (maybe someone else could confirm).
All other data & configuration would match that as at the last backup time.

Presumably them, the 'backup' server becomes the new main server and then the problem is fixed the 'old' can then be the reserve ?  This would save having to try and merge or sync the difference between the two email sets.

Yes & no. Typically you would use the risen server until the main server is repaired & then put the main server back into use, & unrise the backup server. I suggest you do some testing before deploying it. Particularly run through a rise & unrise & see what happens.
See
http://wiki.contribs.org/Affa

Another quick question - is there a quick way to check what contribs I have on the existing v8 server ?

Affa does report installed rpm differences between main server & the backup server.

You can also run
/sbin/e-smith/audittools/newrpms
to get a list of all installed rpms different from the original base version

For other useful diagnostic tools do
ls /sbin/e-smith/audittools

[david000]

janet,

Is the main sme server a Domain Controller (DC)., if so it will have machine accounts for the workstation login (which are unique), so you cannot just create a duplicate server, you will have to backup & restore from the main server to the backup server.

No domain controller at the moment, BUT I do expect that we will go that way in the future. If that's the case then it sounds as if Affa isn't helpful ?

It turns out that the boxes I'm using are x64, so we are good on that front.

Thanks for the commands, just the job.

In the first instance I'll see how the upgrade from 8 to 9 goes (which will also confirm my existing back-up regime) and then see how best to manage having a 'spare'.

 

[janet]

david000

No domain controller at the moment, BUT I do expect that we will go that way in the future. If that's the case then it sounds as if Affa isn't helpful ?

Affa will be OK to use as it backs up the accounts database (which contains the machine accounts).

[guest22]

just as a reference, in a previous decade, I created this http://distro.ibiblio.org/smeserver/contribs/rmitchell/smeserver/howto/High%20Availability%20How-To%20for%20Linux%20Mitel%20SME%20v5.htm


It's old and very outdated, but I wonder if would have covered the requirements...?

[david000]

I spent my Sunday installing and upgrading to v9.  All things considered everything went very smoothly and as advertised. The initial x 64  install went ok as did the eventual restore.

A few comments to help others trying the same thing:
1. To restore back from your backup, the new machine has to be names exactly as the original machine.
2. After rebooting, of course the 'fresh' machine comes up on the same IP as the old....
3. To save the 'fresh' machine collecting mail I 'broke' the gateway IP prior to doing the restore.

I wanted to do the restore offline, form a USB hard drive. However I could not work out how to mount it so had to connect to the Lan and restore from the NAS backup location.

What was impressive was the speed of the restore, maybe 15 or 20 minutes (which sort of makes me wonder if its all there!). 

I also noticed the RAID capability so will double up the drives to get full redundancy.

RequestedDeletion, from what I read it looks like it covers everything. It looks like you do it via serial cables ?  I'll read further later.

janet, Cheers, I'll read up on Affa.  I assume that the usual SME-Backup will create a complete machine, including ibays, but less contribs ? 

One last question - is there anything specific to double check to ensure everything has come over to the 'fresh' V9 machine OK.  I intend to swap it to become the 'Live' Machine next weekend, doing a final restore then.

Thanks again for all your help & suggestions.

   

[janet]

david000

Before doing a backup on the server (immediately prior to switchover to a new server), you should disconnect all workstations (LAN cables or the hub cable), disconnect the WAN cable (to router/Internet), so that no one (external or local or web related) can make changes to the server after the backup is done.

Re speed, that might be OK for a fast server & fast LAN/USB connection with little data, but it is very quick if you have hundereds of Gb on your server hard drive, so ...... I would be checking ?

Did you Verify your backup after it completed ? (to make sure it was a good backup) 

Did you wait for the message Restore is complete, before proceeding (it can take a long time before that message appears even though it seems the restore has finished)

Remember you still have to reinstall Contribs & make sure you use sme 9 versions, & make sure that compatible contribs exist for the essential contribs you already use. Many/most of the sme8 contribs will not work on sme9 due to different path requirements for some items.

You do not tell us how much data (used space) on the drive, or CPU speed/RAM etc of the old & new servers, so it is hard for us to comment.

Re HA Howto.
That is a very old & no longer applicable howto, RequestedDeletion posted it for information only.
DO NOT follow it, but be aware others have attempted to create what you were asking for.
AFAIK Hsing Foo did or suggested some more work using newer software tools (was it Zen ?) but as far as I know dropped his efforts.
If you Google heartbeat & high reliability etc you may see work others have developed or attempted. It is a complex beast so tread carefully, it is not a simple thing to implement (or someone would have already done it for sme server), & the database structure of sme server makes it harder/difficult to achieve.

I assume that the usual SME-Backup will create a complete machine, including ibays, but less contribs ?

Yes, but that is also what Affa will do (just a different underlying procedure)
Here is some additional info re backups, it is worth reading to help you understand what a backup & restore really does in different scenarios
http://wiki.contribs.org/Backup_server_config
in particular
http://wiki.contribs.org/Backup_server_config#Backup_and_Restore_concepts.2C_issues_and_other_information
You may want to add /opt to the backup inclusions, refer wiki Contrib & Howto articles re this

Re restored data.
Yes check everything, users, ibays, data in the ibays, mysql databases, websites, workstation login & user access, contribs working correctlky (after reinstall) & so on. The basic server should be OK from the restore, but add ons & extras & custom templates are where there are likely to be issues, but these can usually be fixed on a "one at at time basis".

I intend to swap it to become the 'Live' Machine next weekend, doing a final restore then.

Final full restore should only be done to a clean fresh installation of sme server from the CD/DVD.  You should not do additional restores over the top of an already done (prior) restore. You can end up with inconsistent folders & data etc.

So reinstall the OS from CD & then restore from the latest backup that you do immediately prior to switchover (maybe overnight if it is big).

Treat what you have done as a test only to prove you are doing it correctly.

[david000]

Hi Janet,

According to 'disk usage' I'm backing up \ restoring about 8gb over a cat5 Lan.

I've installed a new 1tb WD Red (I'm going to double that up before going live) so plenty of space.  It is an older x64 processor with 4Gb RAM.  (Was an XP box).

I did wait for the OK, and it rebooted on the button so I'm pretty sure it's probably OK. It doesn't take much time to verify a backup.

Thanks for the notes on restoring and needing to do a fresh install. That's a shame, but not the end of the world and means that Affa is more appealing.

Point taken on the how to. I'm still very much learning....

Best

David


 

 

[janet]

david000

That amount of data is small so 20 minutes seems OK.

[mercyh]

I keep my AFFA box and my PRODUCTION server at the same level of updates and Contribs. (I actually use the backup box to install new updates or contribs first so I can see what kind of downtime I can expect on the production server when I install them there.) This means the rise is all that needs to happen to create the clone.

BE AWARE THAT IF YOU HAVE DATA IN NON STANDARD PLACES LIKE /OPT, YOU WILL NEED TO MANUALLY ADD THEM TO THE AFFA BACKUP.

I am running my main box as the DC. (these are both on Version 8.1 at this time but I am getting ready to start testing with 64bit version 9).

I am running AFFA on a much slower box so using it as the main server would be only temporary while I rebuild the main box. I have never done a production rise on the backup server, only tests. I have done production restore of individual files or directories from the AFFA backup to the main server multiple times when data has been corrupted or users have deleted files they needed later.

[mercyh]

I probably would not use AFFA but I am at a hospital. We run 24hrs a day, never have downtime to work on equipment and seem to always have an URGENT need to be on the server RIGHT THIS MINUTE. In a normal 8-10hr day environment, it might be overkill.

Installing and restoring a backup on SME server is much different than do the same on a Microsoft server. With SME the install takes around 10 minutes, add 30 minutes to install contribs and run updates. Give yourself 30 minutes for the restore from backup (if you truly have a small dataset) and you are 1hr and 10 minutes from a full reinstalled production box.


Now we both know that it would never actually be that fast as you would have to decide the old box was so messed up that you needed to install on something else, round up a spare box, maybe download the current version of SME and burn it to disk, etc.

However, I would want a good solid 8hrs minimum to do the same thing with a windows server......

[david000]

Janet that's good to know.

mercyh, thanks for the info.   Based on what we have it's not the end of the world to restore form 'cold' and the experience at the weekend confirmed that it would be possible to get it done in a couple of hours assuming I keep some notes !

The idea of Affa is cool for a couple of reasons. It seems I can keep the back up at the same update\contribs level and so be back up and running pretty quick. If I'm travelling or away from the main office it seems I should be able to do this remotely.  After the 'Raise' the backed up domain controller should kick in with the correct user accounts ?

One more question.

On the initial set up I set the Hostname to sme-server.mycomany.local    as per the on screen note.

It seems that it should really match my website domain - so rather than mycomany.local it should be mywebsite.co.uk

Are there any issues in changing that in the config screen ?   

[mercyh]

The idea of Affa is cool for a couple of reasons. It seems I can keep the back up at the same update\contribs level and so be back up and running pretty quick. If I'm travelling or away from the main office it seems I should be able to do this remotely.  After the 'Raise' the backed up domain controller should kick in with the correct user accounts ?

That is correct. The backup includes the user accounts. You do have to get the old machine offline before you bring the new one up as you will have duplicate IP address and host name if you do not.....

[david000]

I've just tried to do a final upgrade from 8 to 9 following all the advice above.  It seemed to go OK, but when started the server I'm getting a cron alert email and mail does not seem to be collected. I've killed the fetchmail, rebooted, reconfigured, deleted likely mail in the server but to no avail.

email content below:
Subject    Cron <root@sme-server> /etc/startmail

fetchmail: another foreground fetchmail is running at 22757.
fetchmail: another foreground fetchmail is running at 22757.
fetchmail: another foreground fetchmail is running at 22757.
fetchmail: another foreground fetchmail is running at 22757.


I've switched off the 'new v9' server and reverted to the 'old' the the cron issue doesn't reappear.  However in fairness we have seen it from time to time over the last few weeks and to finally resolve it I had to delete out the existing mail in our pop boxes, which I'm reluctant to repeat until I can confirm all is ok with the upgrade.

Looking at the pop mail boxes there is very little new mail and nothing huge.

Can anyone help on how to ID the issue ?   

Cheers

David.

EDIT:  I'd forgotten about a legacy pop a/c that was set up to be serviced by fetch mail - I deleted all the spam via web mail and all is well.  The v9 server is up and running and I'll leave it up and see how we go.

[janet]

David

EDIT:  I'd forgotten about a legacy pop a/c that was set up to be serviced by fetch mail - I deleted all the spam via web mail and all is well.  The v9 server is up and running and I'll leave it up and see how we go.

Yes errors like those with fetchmail are usually always related to big or stuck or corrupted messages in the external mail account.

Better if you can to stop using fetchmail, most/many POP accounts let you forward mail to say an address on your server.
.

[guest22]

I fully agree with Janet. Try switching to plain IMAP mechanisms.

[david000]

To stop using fetchmail but still be able to retrieve mail from an assortment of email accounts, how best to do this ?

Fetchmail does seem to the weak link so far, but it seemed the solution at the time....

[janet]

david000

Setup domain based email on your sme server. User accounts on the sme server are email addresses also.

Forward mail from your external accounts to an email address on your sme server, &/or tell people to use your newly setup sme server email addresses.

sme server has a full mail server built in to it which is functional but you do need to enable access as required in server manager email panel.

[david000]

Cheers Janet, I'll ponder that. At the moment we are on a dynamic ip which I imagine would make that harder but I think we can fix that with the ISP. 

I need to understand better the email configuration options (and meanings!) so more reading beckons

[janet]

david000

sme server has provision for a dynamic IP (in the admin console setup screens), so having a dynamic IP connection to your ISP does not really impact upon your ability to run a mail server.
If your ISP can give you a static IP then that's better.

The main thing you need to do is configure external DNS records for your chosen domain name. The mail server is already functional by default so there is not really much to do to get it working.

[david000]

Thanks Janet.

At the moment 'most' mail is forward to common mail box on one of our web servers and collected from there. Thinking about it we should be able to do the same with the other mail boxes too.

I do like the idea of having the mail on a different box for occasions when things go pear shaped, but then I guess the more steps in the chain, the greater the risk.   

[janet]

david000

If an external mail server cannot deliver mail to your sme servers mail server (eg because your sme is temporarily offline or gone pear shaped), then most conforming mail servers will retry delivery for between 2 to 7 days, so your sme server would have to have a pretty long outage in order to loose mail. Linux is a robust server solution.