Topic: qpsmtpd utility scripts

[dmay]

As soon as mirrors synch you will find in my contrib dir some useful (to me, maybe you) qpsmtpd troubleshooting scripts:

http://mirror.contribs.org/smeserver/contribs/dmay/smeserver/7.x/qpsmtpd-utils/

qpsmtpd-connections = displays a tail list of Accepted Connections
qpsmtpd-monitor = real time /var/log/qpsmtpd/current log viewer
qpsmtpd-search = search qpsmtpd logs

Darrell

[piran]

Warning: this topic has not been posted in for at least 120 days.
Unless you're sure you want to reply, please consider starting a new topic

Considered... it's still to do with qpsmtpd scripts so why not include it here.

Code: [Select]

tail -f -n 9000 /var/log/qpsmtpd/current | tai64nlocal | egrep "logging::logterse plugin"
In the same manner that the inclusion of tai64nlocal demystifies the timestamp string is there
something extra I might include to strip out all the characters from the start of each included
line up to and including the terse log's "` " stuff?

Code: [Select]

currently
2007-08-25 07:36:48.959124500 6272 logging::logterse plugin: ` 207.245.252.8    Unknown rcilx075.fido.ca        <>              require_resolvable_client       902     Reverse ld
2007-08-25 07:36:58.555663500 6275 logging::logterse plugin: ` 81.21.195.74     Unknown krakow.janski.edu.pl    <>              require_resolvable_client       902     Reverse ld
2007-08-25 07:44:13.291672500 6560 logging::logterse plugin: ` 208.200.8.8      user8.tomah.com tomah.tomah.com <>              require_resolvable_client       902     Host named

required
207.245.252.8    Unknown rcilx075.fido.ca        <>              require_resolvable_client       902     Reverse ld
81.21.195.74     Unknown krakow.janski.edu.pl    <>              require_resolvable_client       902     Reverse ld
208.200.8.8      user8.tomah.com tomah.tomah.com <>              require_resolvable_client       902     Host named

----best wishes, Robert

[mmccarn]

Try this:
grep logterse /var/log/qpsmtpd/current |awk -F"[\`\t]" '{$1="";print $_}'

Also, while looking into this I found out about /var/log/qpsmtpd/state which seems to be udpated daily during log rotation:
#
# Most recent logfile
# -------------------
#
# Start  : 2007-08-23 12:54:55
# Finish : 2007-08-24 21:01:54
# Elapsed: 1 days, 8 hours, 6 mins, 59 secs
#
# Total transactions :     12349
# Average tx per hour:       384
#
# Cumulative Totals
# -----------------
#
# Start  : 2007-08-15 03:08:16
# Finish : 2007-08-24 21:01:54
# Elapsed: 9 days, 17 hours, 53 mins, 38 secs
#
# Total transactions :       101943
# Average tx per hour:          435
#
#                                 Most Recent Logfile      Cumulative Totals
# Disposition (plugin)            Total       Avg/Day      Total       Avg/Day
# ----------------------------------------------------------------------------
# dnsbl                            5974  48%     4464      54602  53%     5602
# queued                           2287  18%     1709      15908  15%     1632
# check_earlytalker                2128  17%     1590      19235  18%     1973
# check_smtp_forward                899   7%      671       4185   4%      429
# spamassassin                      558   4%      416       3471   3%      356
# require_resolvable_fromhost       445   3%      332       4060   3%      416
# check_basicheaders                 33   0%       24        131   0%       13
# check_goodrcptto                   17   0%       12        177   0%       18
# virus::clamav                       5   0%        3        135   0%       13
# count_unrecognized_commands         2   0%        1          8   0%        0
# check_spamhelo                      1   0%        0         31   0%        3
# ----------------------------------------------------------------------------
# TOTALS                          12349 100%     9228     101943 100%    10460

[piran]

Good stuff;~)

I found this amendment useful as it cut out the four additional empty lines between records.

Code: [Select]

grep "logging::logterse plugin:" /var/log/qpsmtpd/current | awk -F"[\`\t]" '{$1="";print $_}'
I'm more intent on getting the tail functionality working ie real time display.
Have been trying this variant but it seems to suffer from 'something' not sure what;~/

Code: [Select]

tail -f -n 9000 /var/log/qpsmtpd/current | egrep "logging::logterse plugin:" | awk -F"[\`\t]" '{$1="";print $_}'
Its display never amounts to what is produced by the static snapshot of the earlier grep.
Seems to need more and more n ie lines specified and it STILL doesn't finish up with the full tally.
Almost as though it runs out of resources or memory to continue 'processing'.

----best wishes, Robert

[mmccarn]

I have no idea why, but tail, grep, and awk don't seem to play well together...

Here's a command that may show what you want:

Code: [Select]

tail -f /var/log/qpsmtpd/current |awk -F"[\`\t]" '/logging::logterse/ {$1="";print $_}'

[piran]

Agreed. Maybe it's because awk is a programming language in its own right?
Will have a go with and monitor the actions of...

Code: [Select]

tail -f -n 200 /var/log/qpsmtpd/current |awk -F"[\`\t]" '/logging::logterse plugin:/ {$1="";print $_}'...which achieves a pleasing and useful onscreen information density;~)

----best wishes, Robert

[mmccarn]

I meant to put this in, too - so you see how to include/exclude pieces of the output:

Code: [Select]

tail -f /var/log/qpsmtpd/current |awk -F"[\`\t]" '/logging::logterse/ {print $2,$4,$5,$6,$7,$8,$9,$10}'
Also, you can rearrange the output by moving around the $2, $4, etc:

Code: [Select]

tail -f /var/log/qpsmtpd/current |awk -F"[\`\t]" '/logging::logterse/ {print $7,$8 ": "$2,$4,$5}'
(Now you know everything I know about awk...)

[piran]

That's pretty much all I know too.
Found the man awk file a real head-scratcher;~)
Here's one of the things I wanted - works for me - YMMV.

Recently experienced some unwanted email activity (DDOS).
Believe it was due to a 'Storm' infected botnet but that's irrelevant.
I found it 'awkward' to handle, AOP13 is what I wanted back then...

tail -f /var/log/qpsmtpd/current | awk -F"[\`\t]" '/logging::logterse plugin:/ {print $1="/sbin/iptables -A INPUT -s", $2 "/32 -j DROP # Stealth/AOP13"}'

...a dynamic compilation of immediately usable lines for direct injection into Stealth.

Stealth: a defensive technique I've implemented using the masquerading tables,
where the server just drops off the 'radar' of any so included IP domains.
Note that this is for web site activity as well as for email transactions.
pico -w /etc/e-smith/templates-custom/etc/rc.d/init.d/masq/40DenyAOP13
>>add text
/sbin/iptables -A INPUT -s  192.168.0.0/32 -j DROP # Stealth/AOP13
>>Ctrl-X to save
/sbin/e-smith/expand-template /etc/rc.d/init.d/masq
service masq restart
(to normalise just remove 40DenyAOP13 file, expand template, restart masq)

Hope Stealth/AOP13 helps to get someone else out of a 'situation'.
I'll leave it up to you to guess for what AOP13 stands... ;~)

----best wishes, Robert

Inappropriate use of Stealth/AOP13 *WILL* seriously affect your site's visibility.

[mmccarn]

I, too, have developed a set of scripts to block connections from hosts that I thought were attacking me.  I would scan /var/log/qpsmtpd/* and add all dnsbl or check_earlytalker hosts to iptables manually with a 'denylog' rule.

I never could decide if I was being attacked or if there was another problem.  This post: http://forums.contribs.org/index.php?topic=38285.15 makes it look as though there is an underlying problem that can be fixed by installing the latest updates to various perl modules which appeared on the yum repositories late last week (~8/24/2007).

[cactus]

Also, you can rearrange the output by moving around the $2, $4, etc:

Code: [Select]

tail -f /var/log/qpsmtpd/current |awk -F"[\`\t]" '/logging::logterse/ {print $7,$8 ": "$2,$4,$5}'
(Now you know everything I know about awk...)

As I like (local) timestamp and delivery addresses to be listed as well sometimes I use this version:

Code: [Select]

tail -f /var/log/qpsmtpd/current |awk -F"[\`\t ]" '/logging::logterse/ {print $1,$12,$13 ": "$7,$9,$10,$11}'|tai64nlocalwhich adds a localized timestamp at the beginning of the line and the to address as the end compared to the output of the quoted command.