Koozali.org: home of the SME Server
Legacy Forums => Experienced User Forum => Topic started by: arnoldob on May 03, 2004, 03:00:31 AM
-
Hi All,
My SME 6.0 server crashed. I tried to access the console locally and it let me type the username but froze before the password prompt. I had to cold boot the system. When I looked at the sysmon graphs, just prior to the crash, the system processes showed a huge spike. The memory graph swap file reading showed a huge spike as well. My message log is littered with literally thousands of lines like this:
May 1 01:45:03 server kernel: denylog:IN=eth1 OUT= MAC=00:50:bc:ad:6f:54:00:18:e2:31:b0:8c:08:00 SRC=62.163.52.227
I think this would all indicate some kind of attack, maybe related to the Sasser worm or something similar. Has anyone else seen activity like this? Are there any immediate measures I can take to keep it from bringing the server down?
-
If its only your deny-log that's littered with messages, the indications are that your server is doing its job as its supposed to.
If your hardware is a bit on the lean side, however, it might be overwhelmed by such an attack attempt, hence the 'freeze'. You can take the tall spikes in your graphs as indicators for exactly that.
-
Arnold:
You ommitted the most important part of the logfile line ...
(i.e. the end where src & destination ports are listed :-D )
POST SEVERAL FULL LINES OF YOUR LOGFILE...
The only thing we can say now:
(assuming the SRC=62.163.52.227 is all the same)
--- this is your "attacker" and where you might check/complain (later) ---:
inetnum: 62.163.52.0 - 62.163.55.255
netname: UPC-NMG-CABLE
descr: Chello DHCP
country: NL
admin-c: RC482-RIPE
tech-c: RC482-RIPE
tech-c: HMCB1-RIPE
status: ASSIGNED PA
remarks: Contact abuse@chello.nl concerning criminal
remarks: activities like spam, hacks, portscans
notify: hostmaster@chello.at
mnt-by: CHELLO-MNT
changed: hostmaster@chello.at 20020812
source: RIPE
--------------------------------------------------
a "dialin" (modem, cablemodem) at chello NL/AT...
Usually "logging errors" doesn't shutdown your SME...
What's a SPIKE in activity... in %cpu ?
Regards
Reinhold[/quote]
-
Right now I'm running this on a PII 350 256MB on a MSI MS-6119 (intel BX440 based) motherboard. I built it out of spare parts I hand on hand. :) The memory usage has been a concern. I ordered some more memory to bring it up to 512MB (PC100 memory getting harder to find). Here are some links to some sysmon graphs and my mrtg i-bay. The sysmon0.gif is the memory usage graph.
http://server.bertoncini.net/mrtg/
http://server.bertoncini.net/mrtg/sysmon0.gif
http://server.bertoncini.net/mrtg/sysmon1.gif
http://server.bertoncini.net/mrtg/sysmon2.gif
Here's the entire line from a few messages log denylog entries:
May 3 14:04:36 server kernel: denylog:IN=eth1 OUT= MAC=00:50:ba:ad:7f:54:00:08:e2:31:b0:8c:08:00 SRC=68.217.164.157 DST=24.26.98.34 LEN=48 TOS=0x00 PREC=0x00 TTL=108 ID=41701 DF PROTO=TCP SPT=2212 DPT=1740 WINDOW=64240 RES=0x00 SYN URGP=0
May 3 14:04:37 server kernel: denylog:IN=eth1 OUT= MAC=00:50:ba:ad:7f:54:00:08:e2:31:b0:8c:08:00 SRC=172.186.123.82 DST=24.26.98.34 LEN=48 TOS=0x00 PREC=0x00 TTL=113 ID=23722 DF PROTO=TCP SPT=3429 DPT=1740 WINDOW=64240 RES=0x00 SYN URGP=0
May 3 14:04:37 server kernel: denylog:IN=eth1 OUT= MAC=00:50:ba:ad:7f:54:00:08:e2:31:b0:8c:08:00 SRC=69.162.62.81 DST=24.26.98.34 LEN=48 TOS=0x00 PREC=0x00 TTL=110 ID=29885 DF PROTO=TCP SPT=1105 DPT=1740 WINDOW=64240 RES=0x00 SYN URGP=0
May 3 14:04:38 server kernel: denylog:IN=eth1 OUT= MAC=00:50:ba:ad:7f:54:00:08:e2:31:b0:8c:08:00 SRC=196.36.249.246 DST=24.26.98.34 LEN=48 TOS=0x08 PREC=0x40 TTL=105 ID=59246 DF PROTO=TCP SPT=25223 DPT=1740 WINDOW=64240 RES=0x00 SYN URGP=0
My main concern was just preventing future craches. I had noticed that the webserver, mailserver on my server were unavailable starting the same time the swap file usage went through the roof. When I got in to check out what happened. It would not let me log into the console, maybe I didn't wait long enough to see the password promt. But it's never been a delay before. That's when I cold booted the box. Anyway an ideas out there about what you might do next in this situation?
-
Arnold,
The memory usage has been a concern. I ordered some more memory to bring it up to 512MB (PC100 memory getting harder to find). Here are some links to some sysmon graphs ...
Rest assured (or look at the syslog yourself) your
SME is quite healthy with 256MB
... and more wouldn't have helped ... at least in this case :-o
Here's the entire line from a few messages log denylog entries:
May 3 14:04:36 server kernel: denylog:IN=eth1 OUT= MAC=00:oo:oo:oo:... SRC=68.217.164.157 DST=xx.xx.xx LEN=48 TOS=0x00 PREC=0x00 TTL=108 ID=41701 DF PROTO=TCP SPT=2212 DPT=1740 WINDOW=64240 RES=0x00 SYN URGP=0
Let me summarize this for you:
- your outside nic is eth1 with a unique chip-mac info we all know now :-)
- src= is different in each case - see below
- Road Runner is your provider
- some len&id stuff without merit here
- things happened on TCP
- SPT= diffenent for the examples here
- DPT= 1740 ... they all tried to use your TCP Port 1740
... all in vain since this packet was rejected by your kernel.
Look at the list of addresses - all quite different...
=====
68.217.164.157 OrgName: BellSouth.net Inc.
-----
172.186.123.82 OrgName: America Online
-----
69.162.62.81 OrgName: Adelphia Cable Communications
-----
196.36.249.246 OrgName: The Internet Solution, Johannesburg
=====
It seems that some kid is using other peoples machines to DDOS you !!!
My main concern was just preventing future craches. I had noticed that the webserver, mailserver on my server were unavailable starting the same time the swap file usage went through the roof.
It will go through the roof and IMnsHO you can do very little except "turn of your modem"
... and wait till it's "over"...
unplug eth1 if you hear your HD SCREECHING !
Anyway an ideas out there about what you might do next in this situation?
...start looking for a reason "someone likes you"
(if your ip is reasonably stable that is...)
...if you have a variable ip from roadrunner
do what you need to do to get a new ip (modem on/off ?) ... there's ways to do that automatically
if it's always DPT=1740 we could just "drop that without log"
... but in my experience that doesn't really help :-( (search for drop in this forum).
In any case I doubt it's your hardware and all SME services to the outside will suffer
and to keep at lease internal file server activity stable I'd cut outside ties for a while when such an emergency arises.
! NO NEED TO SHUTDOWN BTW !
Sorry to have you no better
Regards
Reinhold
-
Thanks for all the info on how to look at the log. I guess there not alot things to do except monitor things. I might try Dshield to try and fight back at least as much as possible.
Thanks Again :-)
-
Sorry I forgot tot sign in :-)
Thanks Again
-
Hi Arnold,
One last thought: I do not believe that your machine actually crashed
- it was just BUSY TO THE MAX.
...Would take quite some time before your HD overflows.
In cases like this - isolate the server - (warning users if needed)
...unplug eth0,eth1 ...wait ...listen for HD noise
THEN try to login at console.
If you haven't installed the POWERSWITCH.rpc do it!
Good luck :pint:
Reinhold