Topic: USB Backup killed the server!

[reubanks]

OK, enough with the sensational headlines and on with my tale of woe.

I had a server running 7.2 and attempted to backup to an external USB drive prior to replacing the 80GB drive (9GB free) with a pair of mirrored 320GBs . The backup kept failing with the known "Returned non zero" message, so I ran updates from server-manager to bring the system to 7.3 and that went well. I started the backup (to an external 500GB) and went home prepared to return in the morning. Backup at that time was at 40% completed and looked to be finished in 2-3 hours. No workstations had been logged in at any time during the backup procedure.

When I arrived I found that the network had "stopped working" and had been power cycled by someone about 8 hours after I had left (shortly before I returned). When I checked the server it was giving an error that a certificate was missing. The error kept repeating every couple of seconds. If I hit "Enter" it would give me the login prompt, but then the error would pop up again.

I pulled out an SME 7.3 install CD and attempted to do a repair install onto the drive but I received an error saying that the drive was full.

OK, this shouldn't be too terrible... (yeah right) so I used DD as described several places on the forum and as I had done on other machines to clone the drive from the 80GB to one of the 320s but I couldn't figure out how to expand the image to use the extra space. I was tired of thrashing about, so I went out and purchased Acronis True Image Home as I had heard that it worked for this kind of thing.

Well, sure enough it would be happy to expand the partition to fill the drive. The only problem is that it wants to expand the 101.9Mb BOOT partition instead of the 79GB Data partition. I tried the "backup the partitions separately and restore them separately" trick that I read here, but it won't let me expand the partition, only move it around...

I am NOT a Linux guru. I'm sure I'm missing something obvious, but I can't find it in the docs or through searches. Heck, all I really need to do is clear up enough space to allow a repair install and then I'll upgrade using the "put in a larger drive/mirror/replace original with matching larger" trick that has worked for me in the past on another SME system.

Like I said, I'm not a Linux guru. Please be kind to a poor noob.

Thanks.
Randy

[mercyh]

I have used GParted from the Ubuntu live cd for this. You should be able to mount the drive after booting the live cd, you should then be able to change or delete files. You also should be able to change the partion size on the 320mb drive with it.

(GParted is also available on other smaller distros)

[mmccarn]

Why not boot into rescue mode from your 7.3 CD and delete something from your original hdd so that you are no longer out of disk space? /var/log, for example, seems to have a bit over 300MB of stuff that you should be able to delete...

[janet]

reubanks

Please read all of the wiki articles, FAQ, Contrib notes and Howtos.

http://wiki.contribs.org/Raid#Upgrading_the_Hard_Drive_Size

[reubanks]

Well, how does reading all of these articles help with a crashed/filled hard drive?

I succeeded in getting the drive copied and upgraded, but I'm wondering why backing up the drive caused the original drive to fill up. Also, after I had a functional system again I hooked up the external drive and attempted a backup. It took around 8 hours and reported as successful. Funny thing is that the backup drive was empty! (a small text file, but no backup...) I thought maybe it was a mistake so I replaced the internal drive and did a full install. When I was asked if I wanted to restore from backup I said "yes", but no backup was found on the external drive... I then changed to a different external drive that was ext3 formatted and retried the backup. I was told that no drive could be found. (or whatever the verbage is...)

Thank you for all of the helpful hints, but I'm curious. Has anyone ever actually used any of these utilities (GParted, Acronis, etc.) to expand a data partition? The only way I can find that works is the mdadm, pvresize, lvresize, ext2online series of commands. While these are indeed the documented commands to use and they work well, why do all of the alternate utilities I see being recomended on this and other threads simply not work? None of these utilities seem to recognise the partition type and refuse to do anything more than reposition the partition around the free space and not actually expand said data partition. (I've tried all of these on a fresh install and they simply do NOT work.)

I'll be documenting all of the steps I went through and putting in a bug report in the next couple of days. It may be that I am simply flailing around and not pussing the right buttons, but I have been working with computers and various OS's for a lot of years and it seems that there is something going wrong here.

Thanks again for the help.

[mercyh]

reubanks

I will be the first in line for 30 lashes .

I thought I had used Gparted on SME. I know I have used it on Windows and have used it to recover files from a crashed Fedora Core installation. After your information above I got curious and booted my test server with the Ubuntu CD. You are absolutely correct. I could not mount the drive or do anything with the partition. The filesystem is unknown to Gparted. This could be one downside of installing everything as RAID (even if it is one disk).

I succeeded in getting the drive copied and upgraded, but I'm wondering why backing up the drive caused the original drive to fill up. Also, after I had a functional system again I hooked up the external drive and attempted a backup. It took around 8 hours and reported as successful. Funny thing is that the backup drive was empty! (a small text file, but no backup...)

It sounds to me like you are not backing up on the external drive at all, but back to a mount point on the main server drive..............

  However this could be more drivel like the Gparted snafu.

[janet]

reubanks

..how does reading all of these articles help with a crashed/filled hard drive?

The previous post gave you an answer ie
....boot into rescue mode from your 7.3 CD and delete something.....

Then using the link provided you can increase your hard disk partitions the correct way, rather than using other incorrect ways.

Also if you read the Howto articles you will find useful information about using USB drives for backup,
which if followed carefully will most likely resolve your problems.

http://wiki.contribs.org/USBDisks

[reubanks]

I had read a lot (but not all) of the docs on both of my issues and couldn't find what I was looking for. That's why I originally posted.

I really like using SME, and will continue to do so. What's baffling is the whole USB Backup thing. If indeed the backup went to the main mounted drive then WHY!?! I thought that USB drives were handled differently (or at least had a different naming scheme like IDE has HDx and SATA/SCSI has SDx. If this is indeed the case then why did a write to USBfillintheblank go to SDA? On the newly expanded and functional system, why did an 8 hour or so backup that reports as "SUCCESSFUL" leave an empty USB drive?

How many people are backing up to USB drives every night thinking their data is protected only to find that they have nothing once their server crashes/is stolen/is destroyed in a fire/etc.

I am a hardware geek, not a programmer. I am posting these as real honest questions and am not attempting to flame anyone or anything. I'm sure that all of these functions have been tested ad nausium <sic?>  and have worked in the past. So, what happened in my situation? Why do I consistently fail to get a successful backup on multiple machines with a couple of different 7.x builds using different hardware? (The original machine was a Dell Poweredge SC420, I have also tried this on a couple of 400SCs as well as some generic PCs. All of them seem to be EXTREMELY compatible with SME/CentOS.)

As for the suggestions about GParted/Ghost/Acronis, these are all very useful utilities and am glad to have them added to my software library. The question is, what the heck won't any of them work with these partitions? I know that this is a RAID situation, but so is the 101.9MB boot partition! This whole thing makes me feel like a character in Raving Rabbids 3. (DWAAAHHH!!! -thud- <Bunnies cannot use LINUX!>)

Oh well, at least the present issue has been resolved. The server is happily running mirrored 320Gb drives and I will be looking into backups other than copying all of the data to a desktop drive. (Not the backup routine, the ol' click and drag method... It ain't pretty, but it works.)

Oh well, back to the grindstone.

Randy

[mercyh]

Randy,

I am very curious also about your backup situation. Would you be willing to post what you are using for backup ( Backup with DAR, DAR2, Etc.) and what your target setting is?

I use both of the above (Yes, In real life on real servers  ). I do not use them to backup to usb. I have mine backing up to workstation shares.

Royce

[mercyh]

How many people are backing up to USB drives every night thinking their data is protected only to find that they have nothing once their server crashes/is stolen/is destroyed in a fire/etc.

This is a very good reason for test restores as part of a backup procedure.

[mercyh]

I just did a test backup to USB from the Admin Console (Option #8) on my test server. This worked perfectly on a FAT32 formatted drive. 

BUT.......

The drive was named BACKUP. I could see the drive as /media/BACKUP, when I unplugged the drive the BACKUP directory stayed in /media. I then reformatted the drive as NTFS and plugged it into the SME server. Looking in /media everything looked fine and BACKUP was there. I ran the Backup to USB from the Admin Console and the message said Backup was successful. Of course nothing was backed up to the drive as SME cannot mount an NTFS formatted drive. The backup file was left in the /media/BACKUP directory that still existed from when the drive was mounted.