Topic: File size limitations on e-Smith

[Jim Huneycutt]

I recently tried to run ntbackup from a Windows 2000 professional (service pack 3) client networked to an e-Smith 5.5 (Update 2) server/gateway. Ntbackup fails at approximately the two gig mark with an "unknown media error" while backing up directly to a file on the drive mapped to the e-Smith server. I then tried creating the backup file on the client and copying the resulting 3 gig file from the client to the serve. I used windows explorer, xcopy and ftp - all without success.

Checking the Samba version, the e-Smith server shows 2.2.4. Searching the forums and bug reports here and the Internet did not reveal any conclusive information about this problem. For example, this reference http://www.faqchest.com/linux/samba-l/smb-02/smb-0201/smb-020135/smb02012708_15005.html indicates the same problem with Samba 2.0.9 but indicates version 2.2 fixed the problem.

Perhaps this is a well known problem but somehow I overlooked this limitation in my e-Smith research. Any help for a fix would be appreciated since my client would very much like to backup workstations to the e-Smith server. Splitting the backup file into multiple smaller files with one of the file splitter utilities is not a very good option since some of the workstations do not have sufficient disk space to hold a full backup.

Jim

[Mark]

There is bucket loads of info on this file size limit problem on the forums - have you searched the forums? I believe you will find that the 2 gig limit is an e-smith problem and has been discussed multiple times in regard to backing up an e-smith server to a desktop. Maybe backup up the os and then backup the data as 2 operations (just means 2 scheduled tasks instead of 1)?

cheers

Mark

[Kelvin]

Actually, the 2GB limit is a Linux (*nix ??) problem, not only an e-smith problem (comes from the use of 32 bit libraries, etc.) and also not directly a Samba issue either. As Mark says, there's tons of info about this problem and the forums have lots of discussions particular to SME as well.

Later kernels / file systems have addressed the problem but not the ones that come standard with SME up to version 5.5.

The long and the short of it, you cannot store any single file larger than 2GB on the SME server. You can have many, many files in a folder on the server as long as no single file is greater than 2GB (hence the need to to split large databases into smaller chunks).

Kelvin

[Ray Mitchell]

Kelvin

Thats interesting !
Do you or anyone know if these file size limitation issues are being addressed in v5.6 ?

Thanks
Ray Mitchell

[Ray Mitchell]

Jim
Perhaps your users workstation data could be saved in the users home folder on sme server. That way the stuff is backed up whenever the server is backed up.

Then the workstation backup (assuming it is a system backup without data) may be smaller and less than the 2Gb limit, so your normal w/s backup to the sme will be OK.

Regards
Ray Mitchell

[Jim Huneycutt]

Mark, Kelvin and Ray - thanks for the replies. I did search the forums extensively for file size limitations but apparently did not use the proper key words. For my education if you have a relevant thread on the e-Smith forums that would be very useful to me - I am still learning the best way to search these forums. I also searched the Internet as well and nothing led me to believe that this was a general Linux limitation. It is a pretty severe limitation in that I am offering e-Smith servers as a replacement for NT/Win2k servers which have no problem with large files, and this limit kills the deal in some cases - workstation backups and moderately large database files for example.

Ah, well - maybe next release.

Jim

[Kelvin]

Jim,

Search the forum with the keywords "2GB limit" without the quotes (and remeber to search All Dates not just the past 30 days.

You could also search Google (keywords : 2GB ext2).


Kelvin

[Ray Mitchell]

Jim
Your problem is opposite to the backup size problem that is generally spoken of in these forums. Most users are wanting to backup the sme server to a workstation and run into the 2Gb limit when they do a backup to desktop.
See
http://forums.contribs.org/index.php?topic=15411.msg59192#msg59192
for a recent example and quite illuminating one as well.
Search for backup and select all dates and you will get a lot of results.
Regards
Ray Mitchell

[Robert Harlow]

My site uses W2kPro-SP2 boxes, SuSE v8 boxes and SME v5.5 boxes (server only) over a 100Mb network.

The Dantz Retrospect backup application (running on W2kPro-SP2) now effectively resolves this exasperating 2Gb filesize issue of *nix/SME in its latest iteration (v6). Instead of growing inexorably (60Gb+ in my case) it can be directed to both limit itself to a percentile capacity on the target drive over the network but, most importantly, also automatically split its database files into easily managed 600Mb segments on that drive [aka automatic spanning]

best wishes, Robert

[Jim Huneycutt]

Thanks Robert for the Dantz backup tip. I have checked out their site http://www.dantz.com but I'm curious which version you are using - professional, multi-server, etc. My immediate situation is ten Windows user workstations backing up to one e-Smith server. What would you recommend?

Thanks again,
jim

[Robert Harlow]

Jim

I'm not sure I'm your man to whom you should look - owning up to being a SME-newbie AND linux-newbie:~/  So FWIW...

Have used Retrospect on standalone here for some while. Several W2kPro-SP2 flavoured boxes hooked up with a 100Mb unmanaged switch. These are destined to be migrated to SuSE within a year or two. I've included dual booting capabilities so that I can continue the necessary education that will later allow a full migration of my site away from Micro$oft and into Linux:-)

Recently took on Retrospect's version 6 upgrade on the recommendation of their ever-friendly technical support contact. I *would've* liked to go with the version as it is the only one to include specific Linux support. However it is way too expensive and OTT for my site's needs. Have sent a note to Dantz about the new direction of relative unfriendliness towards Linux users, particularly the SoHo category [mine], but such an opinion is unlikely to be acknowledged - perhaps market forces will out [apologies to the Bard]. So I've had to opt for the version.

The pro version's engine has to run on W2kPro. The thing includes 2 client licenses which can handle both Windows & Linux flavours. At the moment I intend to use it only to backup/duplicate/archive the data repository as opposed to the complete SME box. It's so quick to re-install and the work to re-configure is not much of a chore. Maybe later.

Have been held up rather solidly by a pair of failing SCSI drives in the SME box and a consequential poor-contact PCI slot of its network card [due to vibration]. It was hell to diagnose and, on the way, the SME's somewhat scattergun installer routine switched on my seemingly isolated data drives in a thoroughly unguarded moment of extreme tiredness on my part. It was quite upsetting to see the resultant virgin data landscape when the SME's scattergun installer routine had wreaked its ultimate damage:~/ What added insult to that injury was that it didn't even do a good job... or any job at all. It installed itself onto the biggest slowest IDE drives that were all seemingly isolated in the BIOS (with IDE booting disabled too). Completely and utterly ignored the whole nest of fast SCSI drives. It even put its swap file on that big slow IDE drive. The *safe* hdparm flavours used resulted in a 2.5hr install that normally takes just over 12mins when I manually ensure that the SME's scattergun install routine *finds* what it oughta...

So I am using my backups for real - right at this moment:~/ Very recent cover is a bit patchy, it was the failure of the automatic backups that highlighted what was to become a pretty disasterous episode.

As I said beforehand, backing up -TO- the SME can now be achieved as Retrospect can be organised to use 600Mb chunks instead of its old ever-increasing file. Haven't had much experience of pushing backups to the SME, at the moment my entire energy is being put to replacing the lost data repository that the SME was supposed to safeguard and server out.

Given my meagre knowledge/experience with anything to do with the penguin I will continue with a policy of running the W2kPro-SP2 workstations as autonomous units but whose static and changing data needs are addressed on the SME [aka the site's data repository].

The Retrospect (Professional) engine must run from W2kPro (SP2 in my case, as I have zero intention to install the SP3 despite taking the trouble to download that monster of a file). Backups of the site's data repository on the SME must therefore use a W2kPro terminal's resources. I've been using extra hard drives given their cost. The new v6 is also able to use target drives as well as its old way of a large file on a drive. When things are back to normal I need to look at this functionality. My 16 message long posting, headed over in the General forum here at e-smith, covers the nitty gritty of the developing disaster. At the end I have re-iterated my dislike [!?] of the SME's scattergun installer routine and added my own voice to those requesting the ability to use files in excess of 2Gb.

One of the W2kPro boxes has a nice, but old, Tandberg tape streamer which does adequately for the relatively unchanging segment of the data repository. This, the music mp3 files, is some 40Gb or so. It took over six hours last night to reload the SME with all the mp3 files - over the 100Mb network.

Earlier this morning my replacement SCSI drive was delivered [Quantum/Maxtor Atlas III 10k 18Gb LVD]. It was allowed to warm up in its anti-static bag and then carefully installed into the nest. I -DO- like this drive:-) It was so quiet I thought it was surely a DOA, until I saw a reflection of the LED-bead underneath flashing away! Am now reloading all the data repository stuff onto the separately mounted ibay'd drives in the nest. That's all I have SME actually doing at this early stage of the migration project ie that of making mounted drives in the nest available to the W2kPro-SP2 boxes on the network.

Things looking a lot better now, despite the probably patchy level of restore (just for this last week). Still, not much fee-earning actual work got done, so I can't think that I've lost overmuch:-( Need to plan how this loophole might be avoided. Never lost this much before using my Win boxes [ tongue very much in cheek:-) ]

best wishes, Robert