Topic: Need to clean up HDD to recover space!

[Cyrus Bharda]

Howdy all,

Well finnally it has happened, my poor 2 GB HDD is finnally full, but of what? I have been watching the free space shrink on it ever since I installed it, with Shad Lord's system monitor.

My question is that now it is nearly full, what can I do to recover some space?

I have installed the durep for users and there is hardly anything in there as I have set all the users email clients to remove their email off the server.

I have flushed out the squid cache, not that that had any hug impact either..... freed up about 50 MB.

So what is filling up my HDD and what can I do to stop it, or recover some space?

I have the yearly HDD usage graph on display here:

http://mirror.contribs.org/smeserver/contribs/cbharda/alpha/sysmon-HDD-usage.png

Obviously the spikes are where I have copied big files to and from the server, but I have since deleted them, I only did that to see if I could generate any data on the et0's graph, but none showed up, but that's a seperate problem altogether!

So Anyone got any clues?

Thanks,

Cyrus Bharda

[RayG]

After a few months of operation I noticed the backup size on my 5.5 install climb to a little over 600 meg of "unidentified" files. A quick browse through the various directories show most of the bloat to be older log files.

If you have ftp from outside enabled, you may have space "borrowed" for various file sharing networks. They tend to put their files in hidden directories.

[Michael Smith]

Log files are a big chunk, I betcha, esp. on a 2 GB HDD.  Have a look in /var/log

[Nathan Fowler]

I high-demand server with a pretty good amount of content of two 1.2GB drivers in a RAID-1 mirror.  We're talking web services, e-mail services, web-mail, and a few client sites.  I'm not hosting 400MB movies, or MP3s, or anything along those lines.  About 12 websites.

One thing I did to free up space was to remove the contents of /usr/share/doc (and other, I forget it offhand, perhaps /usr/doc).  While some people may actually have the free time and desire to thumb through the pages of ReadMe's and Squid documentation, I often use the authors website or my best friend, Google, for documentation.  By removing these directories I was able to free about 100 MB.

Additionally, I see no value in keeping the rotated log files because I constantly review my logs.  If you use your server as a web-server and handle a large amount of requests you can quickly eat up 100MB in logs easily by the time they rotate.  I'd remove those rotated logs.

Clearing out /tmp is also a good idea.

Also, I remove any RPM's that I am not using, such as Squid, IMP, Horde, etc.  If you are using Squid, you may want to look at clearing out your cache or adjusting your cache settings to something that is more disk friendly.

I had a large amount of RAM on my system, 256MB, and was only using about 64MB (With samba stopped, Samba loves to cache as much available RAM as possible).  I never did any kind of disk swap, and despite the taboo associated with not having swap space I've never had an issue for over a two years.  I destroyed the Swap partition (about 200 MB), created a RAID partition on it, formatted it, and mounted it to /home/e-smith/files/ibays using /etc/fstab.  Since my users don't really use up that much space in ibays, it was an ideal configuration, allocating about 200MB of other-wise unused space. Some people say swap is absolutely necessary, others say it is only necessary if you don't have the physical RAM to accomodate your system load.  Personal experice has taught me that it isn't necessary as long as the amount of RAM is sufficient.

Like you, I constantly monitor free space and take an aggressive approach when removing unused items.

Hope this helped, and please excuse any typos, my eyes are very tired.

Thanks,
Nathan

[Cyrus Bharda]

Ray, Micheal and Nathan,

Wow, have a look at all the space I just created thanks to all of you:

http://mirror.contribs.org/smeserver/contribs/cbharda/alpha/sysmon-HDD-usage-after-cleanup.png

Thanks cleared up about 500 MB still now it is sitting on 1.6GB not doing any web hosting whatsoever, only about 100 users with emails, no ibays, no mp3's.

Guess its all those rpm's I have added on

Thanks guys,

Cyrus Bharda

[Michiel]

There is a great contrib at http://mirror.contribs.org/smeserver/contribs/saco/contrib/e-smith-durep/. It creates two panels in your server manager that will give the disk usage of user accounts and of ibays.

If run from the command line, durep will give you a tree-like overview of the diskusage of each and every directory. See man durep for some usefull options.

Michiel

[Cyrus Bharda]

Michiel,

That was the first rpm I installed thanks for suggesting it though

Cyrus Bharda

[Walter Padgett]

Good Morning,

Thanks for the information, learned where to clean up my hdd some more. I have noticed that if you have SARG installed, acckk, those web pages that are generated with the stats of the web surfing that your users do eats up a ton of space. I'm going in every week after the weekly is done and wiping out the logs. I think that's what shut me down awhile back.

Later

Wally

[toby]

Surprise surprise my one of my SME boxes (with only a 2GB disk ) doing very little had the same problem....the cause over 1GB of sarg logs