Topic: sysmon show lots of outgoing traffic afterhours

[Jáder]

I' a bit scarred because my server is showing lots of outgoing traffic after hours.
Here you can see one example: http://www.abandonemicrosoft.net/publico/Imagens/ETH1-Traffic.tiff

I've tryed to see what is generating this amount of traffic (all upload bandwidth is being used!) using tcptrack on eth1, but it shows nothing. (or I do not know how to use it to show what I' looking for).
Same thing for eth0.

I used netstat -an to show connections too... and as with tcptrack see nothing using a lot of bandwidth.
I saw lot of connections to 207.46.x.y ... and discovered it was owned by M$.
SO... or my server being used to DDoS to M$ or someone on my network is sending a lot of info to M$.
As I said before: I' SCARED!
see /var/log/message :

Code: [Select]

Aug 15 19:17:20 pantera sshd(pam_unix)[24245]: session closed for user root
Aug 15 19:35:36 pantera dhcpd: Wrote 27 leases to leases file.
Aug 15 21:58:37 pantera dhcpd: Wrote 27 leases to leases file.
Aug 15 21:58:37 pantera dhcpd: DHCPREQUEST for 192.168.124.174 from 00:11:5b:c2:ee:c4 (aux4) via eth0
Aug 15 21:58:37 pantera dhcpd: DHCPACK on 192.168.124.174 to 00:11:5b:c2:ee:c4 (aux4) via eth0
Aug 15 23:00:05 pantera sshd(pam_unix)[5456]: session opened for user root by (uid=0)
Aug 15 23:00:05 pantera sshd(pam_unix)[5456]: session closed for user root
Aug 15 23:00:09 pantera sshd(pam_unix)[5517]: session opened for user root by (uid=0)
Aug 15 23:00:09 pantera sshd(pam_unix)[5517]: session closed for user root
Aug 15 23:00:10 pantera sshd(pam_unix)[5538]: session opened for user root by (uid=0)
Aug 15 23:00:11 pantera sshd(pam_unix)[5538]: session closed for user root
Aug 15 23:00:13 pantera sshd(pam_unix)[5568]: session opened for user root by (uid=0)
Aug 15 23:00:14 pantera esmith::event[5570]: Processing event: pre-backup desktop
Aug 15 23:00:14 pantera esmith::event[5570]: Running event handler: /etc/e-smith/events/actions/generic_template_expand
Aug 15 23:00:14 pantera esmith::event[5570]: expanding /etc/dar/DailyBackup.dcf 
Aug 15 23:00:14 pantera esmith::event[5570]: generic_template_expand=action|Event|pre-backup|Action|generic_template_expand|S
tart|1250388014 66342|End|1250388014 342866|Elapsed|0.276524
Aug 15 23:00:14 pantera esmith::event[5570]: Running event handler: /etc/e-smith/events/pre-backup/S10mysql-delete-dumped-tab
les
Aug 15 23:00:15 pantera esmith::event[5570]: S10mysql-delete-dumped-tables=action|Event|pre-backup|Action|S10mysql-delete-dum
ped-tables|Start|1250388014 343155|End|1250388015 132715|Elapsed|0.78956
Aug 15 23:00:15 pantera esmith::event[5570]: Running event handler: /etc/e-smith/events/pre-backup/S20mysql-dump-tables
Aug 15 23:01:28 pantera esmith::event[5570]: S20mysql-dump-tables=action|Event|pre-backup|Action|S20mysql-dump-tables|Start|1
250388015 133065|End|1250388088 163110|Elapsed|73.030045
Aug 15 23:01:28 pantera esmith::event[5570]: Running event handler: /etc/e-smith/events/pre-backup/S50rewind-tape
Aug 15 23:01:28 pantera esmith::event[5570]: S50rewind-tape=action|Event|pre-backup|Action|S50rewind-tape|Start|1250388088 16
3513|End|1250388088 297919|Elapsed|0.134406
Aug 15 23:01:28 pantera sshd(pam_unix)[5568]: session closed for user root
Aug 15 23:01:29 pantera sshd(pam_unix)[5671]: session opened for user root by (uid=0)
Aug 16 00:19:22 pantera dhcpd: Wrote 27 leases to leases file.
Aug 16 00:19:22 pantera dhcpd: DHCPREQUEST for 192.168.124.177 from 00:18:8b:df:53:c8 (marcia) via eth0
Aug 16 00:19:22 pantera dhcpd: DHCPACK on 192.168.124.177 to 00:18:8b:df:53:c8 (marcia) via eth0
Aug 16 03:11:02 pantera squid[5480]: storeDirWriteCleanLogs: Starting...


[Stefano]

just a question.. are you sure you have not any pc doing p2p?

Stefano

[piran]

Might have a new Torrent... the hours look fishy.
I have a bunch of M$ IPs in that area blocked
for LWP::Simple and stuff.

[piran]

...
I saw lot of connections to 207.46.x.y ... and discovered it was owned by M$.

Some routers have a quality of service functionality.
Try setting QOS so that IP only gets to use the bare
minimum o/g bandwidth available until you sort it out.

[piran]

A user might have volunteered your server for TOR
(anonymising proxy stuff) or has set up an on-line
backup to an M$ cloud by way of experimentation.
Just guesses... Maybe running HTOP would show
you something working away in the background.

[mmccarn]

iptraf or ntop might help you figure out what's up.

[Jáder]

just a question.. are you sure you have not any pc doing p2p?

Stefano

AFAIK there are none.
But do you notice the traffic starts at 11pm (was yesterday, and I was there working till 2pm) As you can see there are low traffic from 10am (when I arrived and shutdown router) to 11pm . I post messages log because of start of traffic being 11pm... thought about some cron job. But cannot find anything.
I also cannot see the traffic using any tools I know... iptraf/tcptrac just show small traffic.

[Jáder]

iptraf or ntop might help you figure out what's up.

That is part of problem, I tried to discover what is generating that traffic using tcptrac and iptraf and see nothing!

[Jáder]

A user might have volunteered your server for TOR
(anonymising proxy stuff) or has set up an on-line
backup to an M$ cloud by way of experimentation.
Just guesses... Maybe running HTOP would show
you something working away in the background.

Hum... that could be possible... but shouldn' I be able to watch this traffic and trace it back to one computer using IPTRAF or TCPTRACK ?

Right now just 2 computers (other than server) are up and running. Just like when I left them yesterday 2pm. So why traffic started 11pm ...
I'm thinking the SERVER is generating this traffic... just I do not know how to see/watch/track source!!

Any ideas?

[piran]

Don't know, I'm not up with that sort stuff, it was
just a suggestion to help you sleuth out your difficulty.
Staying with the router, it should be able to directly
tell exactly where a particular masquerading/NAT
session is being routed... so that'll take it down
to the misbehaving or otherwise workstation.

[Jáder]

Piran

Thanks by your tips.
I have some problems about to use router to track it.
My setup is WEB -> ADSL modem (router mode)--> SME server (2 NICs!)--> 24p managed switch --> LAN

I have attempted find the source of traffic on every point... none shows nothing!
And now, I try to restart squid on server... and traffic dropped!
See it now at: http://www.abandonemicrosoft.net/publico/Imagens/ETH1-traffic-post-squid-restart.png

So... it APPEARS to be a SME server problem. Maybe it's time to report a bug.

[piran]

Another suggestion... someone might have volunteered your
box into a distributed processing net like the ones doing the
search for intelligent life, the weather model thing or the
one running assessments on interactions of pills and potions.
In short something deliberately set up to use out-of-hours
CPU cycles on a benign basis.

[piran]

Racking my memory now... I've seen something like this here
before now. The implementation of the nightly backup went
wrong but it was the CPU utilisation that sky'd and not the
o/g bandwidth. Have you defined (in error) an external
target for some backup or something?

[Jáder]

Racking my memory now... I've seen something like this here
before now. The implementation of the nightly backup went
wrong but it was the CPU utilisation that sky'd and not the
o/g bandwidth. Have you defined (in error) an external
target for some backup or something?

Hum... that looks promising!
I have disabled night backups, but I'm not sure I remove ALL configurations, I think  I just change it to DISABLED and removed external USB drive. This is a piece of my crontab:

Code: [Select]

# run-parts

01 * * * * root run-parts /etc/cron.hourly
02 4 * * * root run-parts /etc/cron.daily
22 4 * * 0 root run-parts /etc/cron.weekly
42 4 1 * * root run-parts /etc/cron.monthly

# logrotate
12 1 */7 * * root       /sbin/e-smith/signal-event logrotate


# Backup.Access.Roan97.mdb
07 9-19 * * mon-fri  root /home/e-smith/files/Backup.Access.Roan97.mdb.sh

# Backup task is disabled
# Workstation Backup task is disabled

02 4 * * * root /sbin/e-smith/check4updates -m

# smeserver-clamscan filesystem scan
12      0       *       *       *       root    /sbin/e-smith/smeserver-clamscan

# dar2
05 12 * * FRI root /etc/e-smith/events/actions/dar2-backup Backup.Site
50 12 * * FRI root /etc/e-smith/events/actions/dar2-backup ISO9000
30 12 * * FRI root /etc/e-smith/events/actions/dar2-backup Roberto
20 12 * * FRI root /etc/e-smith/events/actions/dar2-backup sisROAN

# rsync disabled
Note comments about backup being disabled.
Do you think could be something like that ?
Thanks
Jáder

[piran]

Sympathetic shoulder and ideas man that's me;~)
Fixing issues that's bugzilla.
Personally I wouldn't mess with the crontab directly
as it is template driven (usual comments yah dah).
Just put the dar2 stuff into manual with the control
panel and that does not bypass templates. You'll
get the proper restart actions done too. While you
are in the system manager's control panel double
check the settings for all the backups ~ particularly
the target or destination for each backup.