Topic: Dungog-backup causing 100% load ?

[ask_jeeeves]

Hi,

I am running SME 7.1.3. The last couple of days i have been suffering with my server's CPU running flat out, typicaly 85 - 100% according to emails from sme7admin. I wondered if there is anyway of finding out what it is thats running and using the CPU? I have disabled ClamAV, back-ups etc. I have wondered if it may be Gallery2 which is installed, but i want to try and find out if there is a list i can look at like in Windows.
I'm not sure that this is a bug, more a problem with my server.

Thanks in advance, Rob Miller

[Franco]

Can you do a 'top' command when this is happening?
Email uses clam, and clam seem to be a problem with 7.1.3

[ask_jeeeves]

umm, but it's only just started in the last 48 hours thou???

[Franco]

mine too !  

This is exactly what I'm experiencing:
http://bugs.contribs.org/show_bug.cgi?id=2743

Mail system and my web (Using Dans with scan active) stop working

[dmay]

For clamav issues see also:

http://bugs.contribs.org/show_bug.cgi?id=2349

Bandaid to clear and rebuild clamav mirrors.dat:

sv d freshclam
rm -f /var/clamav/mirrors.dat
sv u freshclam

Darrell

[ask_jeeeves]

For clamav issues see also:

http://bugs.contribs.org/show_bug.cgi?id=2349

Bandaid to clear and rebuild clamav mirrors.dat:

sv d freshclam
rm -f /var/clamav/mirrors.dat
sv u freshclam

Darrell

Thanks dmay,
What does this do exactly? I have run this but i still have a loaded CPU.
I now find it hard to use server-manager & putty as the server freezes for serveral minutes at a time now!!!!

[dmay]

I have run this but i still have a loaded CPU.

As was mentioned above, run top (or better yet htop) and see what exact processes are at fault.

Darrell

[ask_jeeeves]

how do i do that?

[christian]

Log in via root in a shell (for example using putty).

at the command line use either the command "top" or "htop"

it will give you a dynamic ordered list  of the processes that are using the most CPU resources.

[ask_jeeeves]

Ah, good that works a treat. Thank You
The problem i am now getting is that my CPU is heavily loaded every few minutes, and when it does the server becomes unresponsive. So when i run the 'top' command in putty its all good and CPU usage is typically under 10%.
But when the CPU becomes loaded, the refresh in putty stops, preventing me from locating the process which is using a high amount of CPU Power.

I have the last hour, received messages from sme7admin exactly every 10 minutes warning me of my high CPU load??

One thing i have noticed although this could be unrelated - i am not sure is:

Code: [Select]

/usr/bin/diff -eN /var/tmp/messages/ /var/log/messages

This seems to be using around 25% Memory

Thanks again in advance,

Rob Miller

[pfloor]

What are the server's specs?

umm, but it's only just started in the last 48 hours thou???

Did you do/install/update anything 48 hours ago?

[ask_jeeeves]

What are the server's specs?

umm, but it's only just started in the last 48 hours thou???

Did you do/install/update anything 48 hours ago?

Hi, i am running SME 7.1.3
With a Sempron 3000 CPU
512mb RAM
160GB Drive

I havn't done anything i can think of. The last thing i installed was Gallery2 about a month ago.

I have seem a few things about ClamAV? Do you think it's related?

Many thanks,

Rob Miller

[pfloor]

7.1.3 came out about 3 weeks ago.  Depending on what version you updated from, there were some issues that the upgrade didn't finish properly.

Have you tried the update process again to make sure everything is updated?  Let's make sure you are all current and configured properly.

# yum update
# signal-event post-upgrade
# signale-event reboot

Run the post-upgrade and reboot even if nothing installs.  There have been instances that post-upgrade didn't run properly so do it anyway just in case.

[ask_jeeeves]

7.1.3 came out about 3 weeks ago.  Depending on what version you updated from, there were some issues that the upgrade didn't finish properly.

Have you tried the update process again to make sure everything is updated?  Let's make sure you are all current and configured properly.

# yum update
# signal-event post-upgrade
# signale-event reboot

Run the post-upgrade and reboot even if nothing installs.  There have been instances that post-upgrade didn't run properly so do it anyway just in case.

Thanks pfloor,

I have run the above, but i am still suffering with the CPU fully loaded. Any other ideas???

Could it be crond?

Thank you again,

Rob Miller

[pfloor]

I have run the above, but i am still suffering with the CPU fully loaded. Any other ideas???

That rules out an incomplete update.

Could it be crond?

Could be anything.  Why do you suspect that, have you done something?

Please tell us about ANY and ALL customization you have done (including but not limited to):
-Custom Templates
-Manual changes to any configuration files
-Installed Contribs
-Installed Applications


To help jog your memory, there are scripts under /sbin/e-smith/audittools that will audit the system.  The most useful one is /sbin/e-smith/audittools/templates.

[ask_jeeeves]

well, every now and then i notice that it uses 100% CPU power when watching 'htop'

[ask_jeeeves]

I have installed:

awstats
user-panel
user-vacations
dungog.net - backup
                 - Email Management
                 - Domain pseudomyns
sme7admin
system monitor
mailman

Dungog.net's backup RPM was the last one i installed, but i am sure that was about two weeks ago.

[pfloor]

Keep going, I know that isn't everything.  What about Gallery?  What else have you installed?

Lets also see what kind of template customizations you have done as well, run:

/sbin/e-smith/audittools/templates

[ask_jeeeves]

ya sorry, i forget how much is on here. Also Gallery2 and phpbb forum
Also, i can't remember the name, but i have added the calendar, and address book to horde.
I have installed OScommerce too, and have change register globals from off - on, which i think is the only manual change to a template i have done.

/sbin/e-smith/audittools/templates

that action list's ton's off stuff

[ask_jeeeves]

Hi, thanks to everone for helping me on this one.
I have found the culprit!
I have backup installed from dungog.net
I have turned off the automatic backups option, and so far i have not had any warning emails from sme7admin.

Thanks very much

Rob Miller

[pfloor]

ya sorry, i forget how much is on here. Also Gallery2 and phpbb forum
Also, i can't remember the name, but i have added the calendar, and address book to horde.
I have installed OScommerce too, and have change register globals from off - on, which i think is the only manual change to a template i have done.

/sbin/e-smith/audittools/templates

that action list's ton's off stuff

I thought so.  That list of "stuff" is manually added templates and whether they are an addition or override. But most important are the ones market "MODIFIED".  If you have any of those, it means you have edited templated and this is a NO-NO.  Audittools is a very powerful tool!

Hi, thanks to everone for helping me on this one.
I have found the culprit!
I have backup installed from dungog.net
I have turned off the automatic backups option, and so far i have not had any warning emails from sme7admin.

Good, but don't stop there.  You need to notify the author to determine if there is a bug with the contrib.  I will change the subject line to see if snoble chimes in with any help for you.

[pfloor]

Changing subject was:

sme7admin cpu = 100 % Loaded ?
to:
Dungog-backup causing 100% load ?

I have notified snoble.

[Franco]

This is not it!
I do not have Dungog-backup installed in this system.
Does the backup get scanned by clam? Maybe it's so big that is causing you this?
I have the Dungog-Dansguardian installed but clam is the suspicious one:
http://bugs.contribs.org/show_bug.cgi?id=2879

[stephen noble]

> well, every now and then i notice that it uses 100% CPU power when watching 'htop'

dungog-backup (gzip) does use 100% at times
but this isn't anything new

set this setting to nice,
*Priority, nice gives priority to other processes:

-------------------------------

The problem here is very likely clamav related

would you adjust the topic heading please when you know more

[ask_jeeeves]

Thanks, i would expect 100% every now and then, but this was every 10 minutes without fail, lasting a few minutes. In these few minutes the server was unresonsive.

Looking at the settings in the backup options, i didn't have 'nice' enabled to it wouldn't have giving priority to other processes.

Where do you want me to go from here?????

Thanks again.

Rob Miller

[stephen noble]

disable or nice backups to check they aren't the problem

then go to the bugtracker for clamav

[Jean-Philippe Pialasse]

i see you are using SME7ADMIN

that is a great contrib but :

what is the size of each peices of /var/log /message (I mean each one )
(sme7admin do a diff of this (and some others logs) every 5 minutes, an dif you have a bad php application with lot of warning it can take up to 10min toi diff the log (so will always have 2 jobs running and 100% CPU)
problem can seem solve after a logrotate but as the log growth the problem comes back (some recurents PHP errors can make message log from 5Mo to 200 Mo in some minutes or hours)
=> you can solve this checking what is growing your log and correct the guilty application or service

and what amount of data have you on your hard drive?

sme7admin allow a df and a du of your hard drive to know what is the amount of data for mysql , home, squid etc. But if you active this it will do a du every 5 min which is a pain as it use a lot of CPU  (for 40 GB you will use about 20% of load continuously)
=> you can solve this by uncheck the du option in sme7admin options