Topic: Monitoring Mail

[Bud]

guys please can you help

i need to find the following information regarding email for a user

messages from company z was sent to user1@domain.com ( sme 9.1 mail server )

user1 is complaining that 30 messages from company z was sent however user1 only received 4 messages

1. how to i check to see where the other messages went for user1?

2. how do i monitor what mail was sent to all users individually and from where the mail originated from for all the users on the sme 9.1 server

any help greatly appreciated 

[brianr]

all the information you'll need to understand this is held in the qpsmtpd logs:

/var/log/qpsmtpd/

and

/var/log/sqpsmtpd/

However you'll need to have some idea of times when the problems arose, and use tai64nlocal to decode the time stamp of the log entries.

I suggest something like this (untested)

cat /var/log/qpsmtpd/*.s | tai64nlocal | grep "time when the problem occurred"

(assuming your logs have rotated since it happened)

else:

cat /var/log/qpsmtpd/current | tai64nlocal | grep "time when the problem occurred"

Pulling out the "logterse" entries (which are summaries) is also a good trick, so add | grep "logterse" to the above. Beware the logterse lines are long!

[CharlieBrady]

messages from company z was sent to user1@domain.com

The IT administrators of company z would be in the best position to investigate what happened to those messages.

[CharlieBrady]

all the information you'll need to understand this is held in the qpsmtpd logs:

Not necessarily. That depends on whether company z's mail server even made an attempt to connect to the relevant SME server (it's safe for us to conclude that "domain.com" is not actually the domain name of the server).

[Bud]

brianr and CharlieBrady, thanks for your help guys, really appreciated.

questions:
1. can you recommend what are the best contrib(s) for mail stats for per user and possibly the headers of mail per user? ( date received )
2. i need to send the user a log of what emails the user1 received per day/week or month. any ideas on how to compile the log to send to user1 or users?
3. how do i show what was junk mail for user1 or users?
4. what contrib(s) do you recommend to backup all mail messages for each user on the sme 9.1 server?
5. is there a way to use tai64nlocal for showing what email a single user has received/sent per day/week/month?

thanks again

[brianr]

There are no contribs that I know of will fulfill the logging requirements that you mention however the smeserver-mailstats contrib is well worth installing (I would say that wouldn't I - I wrote it?) to get an idea about email, spam, etc.

There is a newer version than the one in the contribs repo, but it needs the updated qpsmtpd as well. which should come with version 9.2 (which is in beta).

Email backup is handled by any of the backup systems (e.g. dar or Affa).

[mmccarn]

Here are some notes on analyzing the existing mail log files:
https://wiki.contribs.org/Mail_log_file_analysis

Here are some notes on getting various stats on how your email is performing:
https://wiki.contribs.org/Email_Statistics

From the second link, this command can be used to list all emails received (or blocked) from a given email address or domain:

Code: [Select]

echo -n "TLD to review: "; read TLD; qploggrep $TLD\> |tai64nlocal |awk '{print $1 " "  $2 "\t" $4 "\t" $5 "\t" $6 "\t" $7}'
Note that if the emails are being blocked by one of the early qpsmtpd plugins (dnsbl, check_earlytalker, require_resolvable_fromhost) your log files will not contain the sending email address, only the sending IP address.

If you know the IP address of the sending mail server, you could use this command to easily figure out if the recipients are deleting or moving the messages after they are received:

Code: [Select]

DAYS=1; echo -n "Days of email to scan [$DAYS]: "; read NEWDAYS; \
find /home/e-smith/files/users -name *$(config get SystemName):* -ctime -$DAYS -exec egrep -H "^Received:\ from\ " "{}" \; |\
grep -v "$(config get LocalIP)" |\
egrep "HELO|EHLO" |\
awk -F"[():]" '{ print $1 "\t" $7}'

[CharlieBrady]

5. is there a way to use tai64nlocal for showing what email a single user has received/sent per day/week/month?

tai64nlocal does nothing but convert from one timestamp format to another (from tai64 time format to human readable local time).

[Jean-Philippe Pialasse]

Smeserver-isoqlog might partly answer your needs.
https://wiki.contribs.org/Isoqlog

[Bud]

guys thank you for your help

i have installed the SMEOptimizer Contrib and are getting good stats from that

i have done a signal-event post-upgrade; signal-event post-reboot

however now i am not receiving any mail for my users 

how do i check where the issue is for not receiving any email from a remote catchall pop3 mailbox

any help greatly appreciated

[Bud]

guys what can i say.

it works!!

thank you to all that helped me, much appreciated 

[Bud]

some more questions

i seem to have a lot of issues with being blacklisted ( http://mxtoolbox.com/blacklists.aspx )

using sme 9.1 server

client has a static ip address

the mail for the client is collected via a catchall account on a hetzner server

i think the blacklisting is due to the following:

1. spam being sent out from a pc(s) / server(s) ?
2. to much mail being sent out via smtp from the sme 9.1 server. ie: more than 300 messages per hour
3. ???

questions:
1. how do i limit outgoing mail from the sme 9.1 server to no more than 290 messages per hour?
2. how do i determine what pc(s) / servers(s) on the lan are sending out spam?
3. how can i get a report emailed to me?
4. should i use a different smtp service provider?
5. how do i limit being blacklisted?
6. is there a way to monitor blacklisting and in the event thereof being sent a report via email?

any help greatly appreciated 

[zatnikatel]

how you tried Wireshark on you network to see where all the email is going by ip address run on a PC Linux or windows pc and set it to scan port 25 and it will list the IP address that using port 25

[Bud]

Server sensing Email Out Issue

Huge amount of mail being sent out to three email addresses from sme 9.2 

presse@filmportal.de
jvcyr2002@yahoo.com
davidjones@live.com

they do not stop being sent out with the result all user mail being sent from users are not being delivered

how do i stop the three email addresses on the server

any help greatly appreciated
 

[Stefano]

first of all, check your clients have no virus/malware

if your SME hosts some web sites, check they are ok

[mmccarn]

If you don't normally talk to any of these domains, or to stop all email to these domains while you debug the problem, you could create 'filmportal.de', 'yahoo.com' and 'live.com' as local domains on your SME server, find and fix the source of the problem, then delete the local domains.

[mmccarn]

It's possible that email to these specific addresses would be dropped if you create qmail's 'badmailto' control file and restart qmail:

Code: [Select]

echo 'presse@filmportal.de
jvcyr2002@yahoo.com
davidjones@live.com' > /var/qmail/control/badmailto
sv t qmail

If that causes problems, or when you're done debugging, remove the file and restart qmail again:

Code: [Select]

rm -f /var/qmail/control/badmailto
sv t qmail

[edit]
No - this doesn't seem to work... (I attempted to block a test email, but could still send to it through my SME server).

[Bud]

Stefano

thank you for your help

i am getting tons of the following in my qmail

----------------------------------------

Qmail message queue, viewed at:
Mon Jun 12 14:42:55 2017
2259620 (8, 8/2259620)
  Return-path: davidjones@live.com
  From: "Mr.David Jones" 'davidjones@live.com'
  To: Recipients 'davidjones@live.com'
  Subject: YOUR IMMEDIATE ATTENTION IS REQUIRED
  Date: Mon, 12 Jun 2017 07:54:15 +0000
  Size: 1008 bytes

2254748 (12, 12/2254748)
  Return-path: davidjones@live.com
  From: "Mr.David Jones" 'davidjones@live.com'
  To: Recipients 'davidjones@live.com'
  Subject: YOUR IMMEDIATE ATTENTION IS REQUIRED
  Date: Mon, 12 Jun 2017 06:47:00 +0000
  Size: 1008 bytes

2257480 (7, 7/2257480)
  Return-path: davidjones@live.com
  From: "Mr.David Jones" 'davidjones@live.com'
  To: Recipients 'davidjones@live.com'
  Subject: YOUR IMMEDIATE ATTENTION IS REQUIRED
  Date: Mon, 12 Jun 2017 07:24:32 +0000
  Size: 1008 bytes

2255955 (0, 0/2255955)
  Return-path: davidjones@live.com
  From: "Mr.David Jones" 'davidjones@live.com'
  To: Recipients 'davidjones@live.com'
  Subject: YOUR IMMEDIATE ATTENTION IS REQUIRED
  Date: Mon, 12 Jun 2017 07:03:42 +0000
  Size: 1008 bytes

2256608 (9, 9/2256608)
  Return-path: davidjones@live.com
  From: "Mr.David Jones" 'davidjones@live.com'
  To: Recipients 'davidjones@live.com'
  Subject: YOUR IMMEDIATE ATTENTION IS REQUIRED
  Date: Mon, 12 Jun 2017 07:12:47 +0000
  Size: 1008 bytes

2254519 (13, 13/2254519)
  Return-path: davidjones@live.com
  From: "Mr.David Jones" 'davidjones@live.com'
  To: Recipients 'davidjones@live.com'
  Subject: YOUR IMMEDIATE ATTENTION IS REQUIRED
  Date: Mon, 12 Jun 2017 06:43:56 +0000
  Size: 1008 bytes

2254026 (3, 3/2254026)
  Return-path: davidjones@live.com
  From: "Mr.David Jones" 'davidjones@live.com'
  To: Recipients 'davidjones@live.com'
  Subject: YOUR IMMEDIATE ATTENTION IS REQUIRED
  Date: Mon, 12 Jun 2017 06:36:45 +0000
  Size: 1008 bytes

------------------------------------------------------

with the result no mail is going out as the isp is blocking any delivery of mail

how do i find out where the message originates from? what pc, hostname, user, ip on the lan ?

any ideas 

[Stefano]

Check qpsmtpd log to see who is injecting those emails

[Daniel B.]

Check qpsmtpd log to see who is injecting those emails

It could be injected in qmail directly by eg, a compromised web application

[Stefano]

Indeed, my bad (I'm on the phone..)
So, Bud, check both qmail and qpsmtpd logs

[CharlieBrady]

Indeed, my bad (I'm on the phone..)
So, Bud, check both qmail and qpsmtpd logs

IMO it will be quicker and more useful to find one such message in the queue and look at its Received: headers.

[CharlieBrady]

with the result no mail is going out as the isp is blocking any delivery of mail

how do i find out where the message originates from? what pc, hostname, user, ip on the lan ?

You need to:

1. stop qmail and qpsmtpd
2. identify the source of the problem and then fix it.
3. clean your mail queue of all the bad messages
4. restart qmail and qpsmtpd

You may need to interact with your ISP and/or with some email blacklists before you will then be able to send email via your ISP.

[Bud]

thank you everyone for your input and support

CharlieBrady

2. identify the source of the problem and then fix it. - that is exactly what i am trying to do. what do you recommend i do to identify the source(s) ?
3. clean your mail queue of all the bad messages - how do i determine where in my mail queue all the bad messages are or do you recommend i do the following ?

sv d /service/qmail
mv /var/qmail/queue /var/qmail/queue.spam
yum -y reinstall qmail
signal-event email-update

any help appreciated 

[Daniel B.]

Look at your logs (/var/log/qmail/current and /var/log/qpsmtpd/current), you should be able to identify where those emails are comming from. You can also check the raw message. Eg, identify one of those spam in qmail queue:

Code: [Select]

grep -r 'IMMEDIATE ATTENTION' /var/qmail/queue/mess

Then, open one of the result

Code: [Select]

less /var/qmail/queue/mess/22/2097530

And look especially at the headers. You should either see the client IP if it was submitted through qpsmtpd, or the uid of the user if injected directly in qmail. Depending on that, you can investigate further.

[CharlieBrady]

sv d /service/qmail
mv /var/qmail/queue /var/qmail/queue.spam
yum -y reinstall qmail

This will give a new clean qmail queue, but you will be left with some valid messages stuck in the old queue. I would recommend you be more selective on deleting messages from the existing queue. But it's up to you...
 

[Jean-Philippe Pialasse]

using qmHandle (https://wiki.contribs.org/Qmhandle_mail_queue_manager)

Code: [Select]

qmHandle -h'davidjones@live.com'
will delete all the emails in the queue with the email davidjones@live.com in headers

[Bud]

guys thank you for all your help, much appreciated

ok so i get this problem every day regarding davidjones@live.com

i check the queue and logs
i do a " grep -r 'YOUR IMMEDIATE ATTENTION IS REQUIRED' /var/qmail/queue/mess "
i get over 9k of mail messages from " davidjones@live.com "
with the result i have to keep deleting the emails as per Jean-Philippe Pialasee: qmHandle -h'davidjones@live.com' every day

an example of the " /var/log/qpsmtpd/current "

2260492 (6, 6/2260492)
  Return-path: davidjones@live.com
  From: "Barr.David Jones" 'davidjones@live.com'
  To: Recipients 'davidjones@live.com'
  Subject: YOUR IMMEDIATE ATTENTION IS REQUIRED
  Date: Tue, 20 Jun 2017 16:28:57 +0000
  Size: 1008 bytes

an example of " less /var/qmail/queue/mess/6/2260492 "

Received: (qmail 15347 invoked by uid 453); 20 Jun 2017 16:08:18 -0000
Authentication-Results: mysmeserver.com; auth=pass (login) smtp.auth=theresa
Received: from Unknown (HELO [104.192.4.19]) (104.192.4.19)
 by mysmeserver.com (qpsmtpd/0.96) with ESMTPSA (DHE-RSA-AES256-SHA encrypted); Tue, 20 Jun 2017 18:08:18 +0200
Content-Type: text/plain; charset="iso-8859-1"
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
Content-Description: Mail message body
Subject: YOUR IMMEDIATE ATTENTION IS REQUIRED
To: Recipients <davidjones@live.com>
From: "Barr.David Jones" <davidjones@live.com>
Date: Tue, 20 Jun 2017 16:07:57 +0000
Reply-To: davidwhite_chambers@yahoo.com

Attention:

I am David Jones Attorney to the late Engr.Steve Moore who died with his wi=
fe and their three kids in a car accident.

Before his death he had funds valued at $27 Million deposited in a bank

I need your assistance to retrieve the money left behind by my client.

Get back to me for more details.

Thanks

Barr David Jones

please guys i need help on how do i get rid of this problem 

[Daniel B.]

The theresa account on your server is compromised. Change its password ASAP, and then cleanup the qmail queue

[Bud]

Daniel B. thank you for your quick reply

you mentioned " cleanup the qmail queue "

how do i do that?

i have done so, i will monitor and reply

thank you