Koozali.org: home of the SME Server
Obsolete Releases => SME Server 8.x => Topic started by: Michail Pappas on June 14, 2012, 12:44:27 PM
-
Hello all,
I am monitoring an 8.0 (release) server which has a minimal number of contribs (namely a joomla site). I have been given the task of taking backups of it. Data:
* on the same network connected via Gigabit ethernet, a 2Tb NAS exists that I could use to take backups to...
* Connection to this NAS can be made over SMB, NFS or FTP.
* My SME system atm takes about 8Gb (all mounts included). I expect that this will increase at a rate of 2-3Gbytes per year (at most).
Goal: I want to be able backups in such a way that in case of disaster I can perform a complete restore including system, databases, even contribs... Specifically, I intend to take up to daily backups, on a 30 backup rotation. Which would be the easiest way to do it (via contrib or via the built-in mechanisms)?
-
none of the sme ways will backup contribs etc.
-
jameswilson
none of the sme ways will backup contribs...
That's not strictly true.
If you install suitable contribs to ibays they will be automatically included in standard backups.
Despite some users opinions this is quite OK to do.
If you install suitable contribs to /opt/... and include /opt in the backup inclusions, then they will be included in standard backups also.
What reqman is really looking for is a bare metal type backup, but there are no supported sme backup methods doing that.
Mondo did that a few sme versions ago but it is no longer maintained.
The closest reqman will get, is to use a standard sme supported backup, and in the event of a major failure, reinstall the sme OS from CD, restore from backup, and then install any contribs not included in backups (as per above comments). The contribs will use the restored data, so your server will function identically.
Refer
http://wiki.contribs.org/Backup_server_config#Backup_and_Restore_concepts.2C_issues_and_other_information
-
Thank you and especially Mary, for the detailed explanation... And indeed, I was looking for a bare-metal restore. Based on your explanation(s), as well as the FAQ entries, I think that I'll do backups to workstation, keeping some sort of special, extra joomla backup.
-
reqman
I think that I'll do backups to workstation, keeping some sort of special, extra joomla backup.
If you move joomla to an ibay, which is fairly simple to do by moving the code, tweaking the config file & renaming the joomla mysql db (to avoid clash with rpm installed joomla if preferred), it will be a part of your backup automatically, all mysql db's are included too.
Altrrnatively just include /opt or /opt/joomla in your backup, refer Contrib wiki article for e-smith backup re backup inclusions.
If you are doing a seperate manual backup of joomla don't foget to do or include a dump of the mysql joomla database or your joomla backup will be useless.
-
One nasty way to do this is to use hot swap mirror drives, and swap one out regularly. Given the cost of drives, sticking a new bare drive in once a month and re-purposing one of the older backups is feasible. I may be paranoid, but I hate to see rebuilds when both drives have the same data but with different dates, I feel much safer plugging in a bare drive.
-
mike_mattos
One nasty way to do this is to use hot swap mirror drives, and swap one out regularly.....
Yes if you have a two drive RAID1 array, then it's feasible to swap a drive every now and again. It's a method I have used a lot over the years.
Saves rebuilding from scratch a mchine with a lot of modifications. You can update data (carefully) from more recent backups, but that can also be a bit of a fiddle.
Last time I looked though (with earlier sme 7.x releases) I think the issue was with the kernel, which did not support hot swap, so you really needed to power down the machine (even if you used so called hot swap drive caddies or bays). It's only a few minutes downtime on modern fast equipment.
Not sure if sme8 kernel supports hot swap.
Every backup/restore method has it's pros and cons.
-
Not sure about the kernel, but a hardware raid cage allows any drive to power down, 'hot swap' to the next driver, power up and the SME raid monitor seems to figure it out, and if not, command line can force the matter!
There is a 3 drive cage on the market that is great for this, two live and a spare ready to go, each with its own power switch! No messy cables to worry about, just take the carrier out to replace with a new drive for next time.
-
mike_mattos
Not sure about the kernel, but a hardware raid cage allows any drive to power down, 'hot swap' to the next driver, power up and the SME raid monitor seems to figure it out, and if not, command line can force the matter!
Purpose specific specialised hardware RAID may work OK.
I was referring to the software RAID1 in sme server, which in many cases is preferable to "fake" hardware RAID.
Let us know how it goes.
-
As far as I can tell, running a "locate joomla" shows everything is installed under /home/e-smith/files/ibays/Primary/
Therefore, it seems I might be covered, correct?
-
re raid, I use the hardware in dumb mode, so SME does the mirror, the cage just allows the drives to be switched without opening the case or shutting down the server.
The cage holds 3 drives in carriers, typically SATA 1,2,3. SME only ever sees SATA1 and SATA2, the third drive is just stored but not powered on, and moved to the SATA2 slot when the loaded SATA2 drive is removed as a backup.
-
And indeed, I was looking for a bare-metal restore.
Doing a bare-metal restore is not recommended. SME's builtin backup stores configuration and user filesets, and requires a fresh SME server installation as the bare-metal recovery base.
-
Goal: I want to be able backups in such a way that in case of disaster I can perform a complete restore including system, databases, even contribs... Specifically, I intend to take up to daily backups, on a 30 backup rotation. Which would be the easiest way to do it (via contrib or via the built-in mechanisms)?
use something like vmware esx, proxmox ve or another virtualization solution, virtualize your SME and do a full backup of your virtual machine..
this is the only way to achieve what you are asking for..
-
Charlie, I'm confused here. If I put a mirrored drive aside before I test something that may blow up the system, and then replace it, where is the problem?
Obviously the backups made during the test period cannot be restored except on a file by file basis, and the drive should go back to the same system it came from. I can see how restoring an image to a different drive or a different system could be a problem.
-
Charlie, I'm confused here. If I put a mirrored drive aside before I test something that may blow up the system, and then replace it, where is the problem?
No problem. However, that is not a bare-metal restore.
-
mike_mattos
I may be paranoid, but I hate to see rebuilds when both drives have the same data but with different dates, I feel much safer plugging in a bare drive.
Indeed you should (ie MUST) blank any drive BEFORE you reuse it (at least when using sme software RAID), using the dd command or another partition deleting utility such as delpart.exe
If your system rebuilds the array to the older data (due to a pre-existing unknown problem) then all the current data will be lost, and you may not even realise this has happened until some time later.
There was a classic example of this in bugzilla a couple of years ago that Charlie very cleverly tracked down.
-
reqman
.....joomla.....is installed under /home/e-smith/files/ibays/Primary/
Therefore, it seems I might be covered, correct?
If it is truly installed there then it will be included in standard backups
cd /home/e-smith/files/ibays/Primary
ls -al
should confirm
-
OK, so if I make an image of the removed mirror drive, load the image onto a new hard drive, and then put that drive into the chassis, have I met the bare metal restore rules?
Disk Images are getting more interesting now that cloud storage has become so cheap, and download speeds so high, keeping disk images seems like a reasonable thing to do, rather than worry that a CD copy of the right version of SME is going to be available when needed, that the contribs will come back to life if restoring on a different version. I noticed some but not all machines moved from 7.5.1 to 7.6 despite the Software Installer saying they were up to date. I don't have CD's of all the dot versions, but I do have cloud backups of servers, just not disk images.
-
mike_mattos
re raid, I use the hardware in dumb mode, so SME does the mirror, the cage just allows the drives to be switched without opening the case or shutting down the server.
The cage holds 3 drives in carriers, typically SATA 1,2,3. SME only ever sees SATA1 and SATA2, the third drive is just stored but not powered on, and moved to the SATA2 slot when the loaded SATA2 drive is removed as a backup.
Are you saying that you already use this configuration and it works OK ?
Are you saying that sme is configured for software RAID1 ?
In a true hardware RAID, sme would only see your hardware RAID drives as a single drive.
Are you saying you can remove a drive, replace it with a blank drive, and the sme admin control panel for drive management will see this new drive and add it to the array when prompted, all without powering down sme, and this is on sme8.0 final ?
-
The drive cage is called an SAS/SATA backplane, supports software raid 0,1, and with a RAID controller, Raid 5. Can hold 3 live drives, but I use 2 live with the third bay only as a place to store the spare.
Yes, I've hot swapped drives with SME7, using SME software RAID1 possible an 8 beta but not sure.
At least once it worked with no intervention, no powering down.
-
mike_mattos
I noticed some but not all machines moved from 7.5.1 to 7.6 despite the Software Installer saying they were up to date.
That was probably caused by not correctly updating the servers with the new repository locations due to Centos 4 EOL. There were about 3 "corrections" needed IIRC, refer sticky forum thread.
-
I had a bad case of Monday morning, and rather than do a controlled drive swap, managed to actually induce a raid error by resetting the drive cage rather than powering down drive 1. The error would not self correct, either before or after I did a drive replacement.
The good news is, the drive cage allowed BOTH HARD DRIVES to be powered down at the same time, then powered up, and SME Server 8.0 continued working and did not require a reboot. However, I didn't want to do a manual repartition and rebuild, so I rebooted the system. I then used the admin panel to instal the replacement drive which had been automatically detected.
I'll try again after the array rebuilds and I've had more coffee to just swap drives around, but the major point is, the hot swap feature worked allowing drive replacement without opening the case or powering down, while the server continued to function.
-
Having used both Ghost and Acronis, I prefer Acronis...personal preference. The features and price point are great.
Yes, there are more granular ways of making backups...as discussed previously. Can't beat Acronis for bare-metal. Not sure why it isn't recommended - maybe someone can advise?
-
use something like vmware esx, proxmox ve or another virtualization solution, virtualize your SME and do a full backup of your virtual machine..
this is the only way to achieve what you are asking for..
I agree with Stefano.
At this moment I have 3 virtualized installations of SME Server.
First is running 2 years without any problems on Vmware ESXi.
The two others are on Proxmox VE 2.1 and working great.
I chose Proxmox due to very good performance and outstanding bare-metal backup functionality with snapshots.
This is the disk read performance of th virtualized SME Server on Proxmox (Quad core Xeon 2.0GHz, 6GB RAM, 1TB Samsung SpinPoint drive's in Raid10):
[root@server05 Panel]# hdparm -tT /dev/hda2
/dev/hda2:
Timing cached reads: 12532 MB in 2.00 seconds = 6274.26 MB/sec
Timing buffered disk reads: 1668 MB in 3.02 seconds = 551.76 MB/sec
Write performance:
[root@server05 Panel]# dd if=/dev/zero of=test bs=1M count=1000
1000+0 records in
1000+0 records out
1048576000 bytes (1.0 GB) copied, 4.05718 seconds, 258 MB/s
Snapshot backup of running SME Server virtual machine with 350GB virtio disk:
[root@server05 Panel]# df -k
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/hda2 349901188 4873644 326966876 2% /
/dev/hda1 101018 19680 76122 21% /boot
tmpfs 2075824 0 2075824 0% /dev/shm
Vzdump:
INFO: starting new backup job: vzdump 200 --remove 0 --mode snapshot --compress lzo --storage zfs --node proxmox
INFO: Starting Backup of VM 200 (qemu)
INFO: status = running
INFO: backup mode: snapshot
INFO: ionice priority: 7
INFO: Logical volume "vzsnap-proxmox-0" created
INFO: creating archive '/zfsfast/dump/vzdump-qemu-200-2012_07_13-10_46_22.tar.lzo'
INFO: adding '/zfsfast/dump/vzdump-qemu-200-2012_07_13-10_46_22.tmp/qemu-server.conf' to archive ('qemu-server.conf')
INFO: adding '/mnt/vzsnap0/images/200/vm-200-disk-1.qcow2' to archive ('vm-disk-ide0.qcow2')
INFO: Total bytes written: 17344956928 (66.43 MiB/s)
INFO: archive file size: 3.67GB
INFO: Finished Backup of VM 200 (00:04:26)
INFO: Backup job finished successfully
TASK OK
Works great!
Eryk