Topic: Server crashed, processor at 100%

[sebahot]

I'm using SME 6.0.1 and from time to time (no particular pattern) server is crashing and processor stays at 100%. I can not login from the console.
Please tell me how can I see what is going on and how to solve this problem.

Thank you,

Sebastian

[cc_skavenger]

Need more info, need a description of the hardware; processor speed, amount of memory, hard drive size, etc.

Also need to know what contribs you have added,
spam filter, clamav, etc.

How is the server setup, server-only or server/gateway?

Is this server used for e-mail?

Just some questions that will help in trying to figure out what is going on.

[sebahot]

Hardware:
Intel PIII E 1GHz
256MB sdram
40GB 7200rpm IDE ExcelStor Technology J640

Contribs installed:
spamassassin-2.53-1
clamav-es-0.85.1-es01
e-smith-squidGuard-0.2-3
e-smith-portforwarding-0.2.0-02
awstats-sme-server-6.3-1
e-smith-sarg-1.2.3-2
snort-2.1.1-1
sme-snort-2.3-2
sme-acid-0.2-1

The server is set-up in server-gateway mode.

I use the server for email but no emails have been received or sent on the time when this happend.

I have noticed this high processor ocuppancy after instaaled Snort+Acid. But, the server have been ok for two days, only today this occured.

If I can give some more details, please tell me.


Sebastian

[cc_skavenger]

how much free space is there on the hard drive?
df -h

Are you able to leave the console on with top running?  I am wondering if spamassassin or snort is causing the problem.  Spamassassin is notorious for hogging memory and cpu.

[sebahot]

# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/hda3              35G   21G   12G  62% /
/dev/hda1             995M   14M  930M   2% /boot
none                  125M     0  124M   0% /dev/shm

I've stoped Snort and the processor activity is low ~10%, but on the other hand the sistem worked fine for 2 days with all services running.

Here is the 'top' results:

 11:02pm  up  5:01,  1 user,  load average: 0,71, 0,72, 0,58
154 processes: 145 sleeping, 8 running, 1 zombie, 0 stopped
CPU states:  2,9% user,  7,3% system,  0,0% nice, 89,6% idle
Mem:   255584K av,  250892K used,    4692K free,       0K shrd,   49204K buff
Swap: 1052248K av,   96992K used,  955256K free                   87572K cached

  PID USER     PRI  NI  SIZE  RSS SHARE STAT %CPU %MEM   TIME COMMAND
 4754 root      10   0   224  196   148 R     0,5  0,0   0:27 ipfm
   13 root       9   0     0    0     0 SW    0,3  0,0   0:07 kjournald
 4745 squid     10   0 17276 8920   712 R     0,3  3,4   1:24 squid
 4475 root      10   0  7276 3992   784 S     0,1  1,5   1:08 sme6admind
 3880 root      10   0  1064 1064   792 R     0,1  0,4   0:00 top
 3979 root      10   0  2856 1432   596 S     0,1  0,5   0:00 sysmon
 3995 root      16   0   996  996   848 S     0,1  0,3   0:00 sh
 3996 root      16   0   540  540   460 S     0,1  0,2   0:00 ping
    1 root       8   0   392  352   332 S     0,0  0,1   0:05 init
    2 root       9   0     0    0     0 SW    0,0  0,0   0:00 keventd
    3 root       9   0     0    0     0 SW    0,0  0,0   0:00 kapmd
    4 root      19  19     0    0     0 SWN   0,0  0,0   0:00 ksoftirqd_CPU0
    5 root       9   0     0    0     0 SW    0,0  0,0   0:20 kswapd
    6 root       9   0     0    0     0 SW    0,0  0,0   0:04 kscand
    7 root       9   0     0    0     0 SW    0,0  0,0   0:00 bdflush
    8 root       9   0     0    0     0 SW    0,0  0,0   0:00 kupdated
    9 root      -1 -20     0    0     0 SW<   0,0  0,0   0:00 mdrecoveryd
  104 root       9   0     0    0     0 SW    0,0  0,0   0:00 khubd
  249 root       9   0     0    0     0 SW    0,0  0,0   0:00 kjournald
  400 root       9   0   368  316   312 S     0,0  0,1   0:00 mingetty
  401 root       9   0   368  316   312 S     0,0  0,1   0:00 mingetty
  402 root       9   0   316  304   260 S     0,0  0,1   0:00 svscan
  407 root       9   0   296  252   248 S     0,0  0,0   0:00 supervise
  408 root       9   0   296  252   248 S     0,0  0,0   0:00 supervise
  411 root       9   0   284  232   228 S     0,0  0,0   0:00 cvm-unix-local
  412 cvmlog     9   0   316  260   256 S     0,0  0,1   0:00 multilog
  420 root       9   0   296  252   248 S     0,0  0,0   0:00 supervise
  421 root       9   0   296  252   248 S     0,0  0,0   0:00 supervise
  422 root       9   0   296  252   248 S     0,0  0,0   0:00 supervise
  423 qmaill     9   0   312  260   256 S     0,0  0,1   0:00 multilog
  424 root       9   0   292  252   248 S     0,0  0,0   0:00 supervise
  425 qmaill     9   0   312  260   256 S     0,0  0,1   0:00 multilog
  426 root       9   0   292  252   248 S     0,0  0,0   0:00 supervise
  428 root       9   0   292  252   248 S     0,0  0,0   0:00 supervise
  429 dnslog     9   0   332  292   276 S     0,0  0,1   0:05 multilog
  430 root       9   0   292  252   248 S     0,0  0,0   0:00 supervise
  431 root       9   0   292  252   248 S     0,0  0,0   0:00 supervise
  435 root       9   0   292  252   248 S     0,0  0,0   0:00 supervise
  436 root       9   0   292  252   248 S     0,0  0,0   0:00 supervise
  449 root       9   0   296  252   248 S     0,0  0,0   0:00 supervise
  450 root       9   0   296  252   248 S     0,0  0,0   0:00 supervise

Do you think Spamassasin is the problem, maybe disable it and enable snort to see what is happening?

Sebastian

[cc_skavenger]

I would put my money on Spamassassin.  Try disabling it and see how things run.

[raem]

http://mirror.contribs.org/smeserver/contribs/rmitchell/smeserver/howto/Mail%20system%20tweaks%20HOWTO%20for%20sme%20server.htm

[sebahot]

I've checked Spamassasin and it was disabled when this happened.

# /etc/init.d/spamassassin status
spamd is stopped

Maybe is because of Snort+Acid?


Sebastian

[cc_skavenger]

it is possible.  If you have a spare server that you can use, I would backup this server and restore it on another server and rebuild it.  I would leave contribs out that you might think are causing the problem.  If it stays stable, then start adding on the extra contribs, one at a time.  Give them some time to make sure that they don't break things.

just a suggestion.