Koozali.org: home of the SME Server
Obsolete Releases => SME 7.x Contribs => Topic started by: Knuddi on August 26, 2008, 11:28:23 AM
-
The updated information on how to use the UnJunk Manager is now updated on the Wiki at: http://wiki.contribs.org/Sme-unjunkmgr
SME UnJunk Manager has been updated to 1.0.1-1 with the following changes:
http://wiki.contribs.org/Sme-unjunkmgr#Revisions
I have for a long time had ”customers” that POP’ed emails from my servers and that therefore did not have a good chance to see and/or retrieve emails that had been sorted to the junkmail folder by SpamAssassin.
I have therefore made a junkmail manager that on a weekly basis sends out a summary email to all users with a list of emails that have been sorted into the junkmail folder the recent week.
If the user wishes to get one of these emails sent to the users inbox he/she can push the “UnJunk” link to the left. The system will, if UnJunk is pressed, teach SpamAssassin via the “sa-learn” command that this can be considered as “ham” next time. This means that it’s more likely that an email from this sender or with this content will pass the spam check next time.
(http://sme.swerts-knudsen.dk/images/emailsummary.jpg)
Installation:
The package needs to Perl-MIME-Lite package to be able to send out HTML formatted emails.
yum install perl-MIME-Lite --enablerepo=smecontribs
Then install the sme-unjunkmgr RPM from (I will get around and release to contribs later):
rpm –Uvh http://sme.swerts-knudsen.com/downloads/unjunkmgr/sme-unjunkmgr-1.0.1-1.noarch.rpm
De-installation or de-activation:
You can simply remote the package again with the usual rpm command
rpm –e sme-unjunkmgr-1.0.0-1
or disable the functionality with:
/sbin/e-smith/db configuration set unjunkmgr service enabled yes
Configuration:
The SpamAssassin learning does require that the SpamAssassin bayes filters have been enabled though. How this is done can be read at http://wiki.contribs.org/Email#Setup_Blacklists_.26_Bayesian_Autolearning or through these few shell commands.
config setprop spamassassin UseBayes 1
config setprop spamassassin BayesAutoLearnThresholdSpam 4.00
config setprop spamassassin BayesAutoLearnThresholdNonspam 0.10
expand-template /etc/mail/spamassassin/local.cf
sa-learn --sync --dbpath /var/spool/spamd/.spamassassin -u spamd
chown spamd.spamd /var/spool/spamd/.spamassassin/bayes_*
chown spamd.spamd /var/spool/spamd/.spamassassin/bayes.mutex
chmod 640 /var/spool/spamd/.spamassassin/bayes_*
config setprop spamassassin status enabled
signal-event email-update
The UnJunk Manager also collects statistics on the emails passing through the mail server. This is accessible from:
http://your.domain.com/unjunkmgr
This web page can by default only be seen from the local network (IP ranges defined in Local Network in the server-manager) but if you want this to be accessible from remote networks (public access) this can be done via:
/sbin/e-smith/db configuration setprop unjunkmgr LocalOnly <no|yes>
(http://sme.swerts-knudsen.dk/images/unjunkweb.jpg)
An additional feature of the UnJunk Manager is to send the collected statistics to a central statistics central at http://central.swerts-knudsen.dk. The gathering of data is not stressful for the server and the data sent is not sensitive (well in my opinion) and consists of:
- amount of emails scanned
- number of spam infected or virus infected emails
- if virus infected the name of the virus found
- the version of SME server used
Should you want to disable this functionality then this can obviously be done:
/sbin/e-smith/db configuration setprop unjunkmgr statsclient <disabled|enabled>
Today around ~850 servers still active with my sme-spamfilter contrib and ~450 servers using my sme-antivirus for SME 6x send in data for this “worldwide overview”.
(http://sme.swerts-knudsen.dk/images/centralweb.jpg)
FAQ:
Q: How do I see what this Summary of Junkmail looks like without sending to all my users?
A: You will need to change the “$debug = 0;” to “$debug = 1;” in the configuration section of the /usr/local/unjunkmgr/spamreminder.pl file. Then lauch the reminder manually with:
/usr/local/unjunkmgr/spamreminder.pl
All the summary emails will now be sent to “admin”.
-
Nice How-to, it would be great to have it in the wiki as well.
Regards,
Niklas
-
I will get around and do that if this contrib is being used by others.
-
Very nice Knuddi, I made a test installation, but for some reason, the two files /unjunkmgr/spam.html and /unjunkmgr/virus.html doesn't exist after installation.
Do you have any clues what I should do to fix that?
-
These two should be in /usr/local/unjunkmgr and accessible when you click on the two MRTG graphs. I all cases they are created by MRTG which should run on a hourly basis.
-
Unfortunately not...
They're NOT there, and the don't get created by MRTG, --- wait a minute, who said I need to have MRTG installed and how do I install MRTG on 7.3???
-
ahaa - that might be my mistake. I actually thought that MRTG was part of default installation. I had completely forgotten that (if) I hade installed it. I strongly assume that you can get it via yum though.
yum install mrtg
-
No...
No Match for argument: mrtg
-
I have now tried to convert this into a wiki page:
http://wiki.contribs.org/Sme-unjunkmgr
-
That is odd - I get it directly from "base" repository...
[root@project7 unjunkmgr]# rpm -qa | grep mrtg
warning: only V3 signatures can be verified, skipping V4 signature
[root@project7 unjunkmgr]# yum install mrtg
==============================================================
WARNING: Additional commands may be required after running yum
==============================================================
Loading "smeserver" plugin
Loading "fastestmirror" plugin
Loading "installonlyn" plugin
Setting up Install Process
Setting up repositories
smeaddons 100% |=========================| 951 B 00:00
updates 100% |=========================| 951 B 00:00
base 100% |=========================| 1.1 kB 00:00
smeos 100% |=========================| 1.9 kB 00:00
smeupdates 100% |=========================| 1.9 kB 00:00
Loading mirror speeds from cached hostfile
Reading repository metadata in from local files
Excluding Packages from CentOS - updates
Finished
Excluding Packages from CentOS - os
Finished
Parsing package install arguments
Resolving Dependencies
--> Populating transaction set with selected packages. Please wait.
---> Downloading header for mrtg to pack into transaction set.
mrtg-2.10.15-2a.i386.rpm 100% |=========================| 57 kB 00:00
---> Package mrtg.i386 0:2.10.15-2a set to be updated
--> Running transaction check
Dependencies Resolved
=============================================================================
Package Arch Version Repository Size
=============================================================================
Installing:
mrtg i386 2.10.15-2a base 915 k
Transaction Summary
=============================================================================
Install 1 Package(s)
Update 0 Package(s)
Remove 0 Package(s)
Total download size: 915 k
Is this ok [y/N]:
-
Are we talking about SME 7.3 here?
This is what I get:
yum install mrtg
==============================================================
WARNING: Additional commands may be required after running yum
==============================================================
Loading "fastestmirror" plugin
Loading "installonlyn" plugin
Loading "smeserver" plugin
Setting up Install Process
Setting up repositories
Loading mirror speeds from cached hostfile
Reading repository metadata in from local files
Excluding Packages from CentOS - updates
Finished
Parsing package install arguments
No Match for argument: mrtg
Nothing to do
================================================================
No new rpms were installed. No additional commands are required.
...and after a
yum clean all:
[root@server ~]# yum install mrtg
==============================================================
WARNING: Additional commands may be required after running yum
==============================================================
Loading "fastestmirror" plugin
Loading "installonlyn" plugin
Loading "smeserver" plugin
Setting up Install Process
Setting up repositories
smeaddons 100% |=========================| 951 B 00:00
updates 100% |=========================| 951 B 00:00
smeos 100% |=========================| 1.9 kB 00:00
smeupdates 100% |=========================| 1.9 kB 00:01
Loading mirror speeds from cached hostfile
Reading repository metadata in from local files
primary.xml.gz 100% |=========================| 186 B 00:00
primary.xml.gz 100% |=========================| 207 kB 00:02
updates : ################################################## 655/655
primary.xml.gz 100% |=========================| 248 kB 00:02
smeos : ################################################## 529/529
primary.xml.gz 100% |=========================| 94 kB 00:01
smeupdates: ################################################## 149/149
Excluding Packages from CentOS - updates
Finished
Parsing package install arguments
No Match for argument: mrtg
Nothing to do
================================================================
No new rpms were installed. No additional commands are required.
-
Yes this is 7.3 clean install. Check your /etc/yum.conf to see whether [base] is enabled
[base]
enabled=1
mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=os
name=CentOS - os
gpgcheck=0
enablegroups=0
exclude=kernel kernel-smp kernel-xenU mkinitrd mdadm initscripts
-
Stupid me... :-? :-? :-?
Of course it wasn't enabled (for some reason...)
Now it works a LOT better.
Sorry for all the fuzz....
-
nice,
i test it
dave
-
Knuddi
I´ve tryed to write you about unJunkMGR using "contribs --#AT#-- swerts-knudse--#DOT#-- dk
but got: "Technical details of permanent failure: DNS Error: Domain name not found"
I was using my gmail account. Do you have a problem?
Here was 90% of my e-mail:
I LOVE your contrib... have lots of ideas about it:
1) I´d love to use the spam statistics generated to add on Zabbix monitoring... will try later
2) I´ve problems with graphs... I´m getting 404 error: "Not Found - The requested URL /unjunkmgr/spam.html was not found on this server."
but I just installed it yesterday! :)
3) I´m using w/debug=1 (for now) but when running the "/usr/local/unjunkmgr/spamreminder.pl" I´m getting two e-mails... MAYBE because I have a lot of spam... there are a limit to add for one e-mail?
4) I´d love to translate the e-mail sent. For now I updated that file for Brazilian Portuguese! ;) Could the messages be on a template file or something else to avoid overwrite on update ?
5) I saw your Google map with PINs for use or your CLAMAV/SPAMASSASSIN contrib... and miss my servers in Brazil OR I´m only user in Brazil!
:(
6) the graphs fo spam and virus couldn´t be at vertical style... so left column would be SPAM and right column VIRUS ?
Thanks
Jáder
-
Hi Jader,
Thanks for the feedback. I had written my email address wrongly - it was missing an "n" in the end of knudsen.
ad 2) The package requires the MRTG package which was missing from the original howto. This can be installed easiyl via:
yum install mrtg --enablerepo=base
ad 3) The two emails you get, are they identical? You should only get one email to admin but per user that it finds junkmails newer than 1 week.
ad 4) Good idea - I will add this to the list
ad 5) I only run this once a week - takes a long time to generate and it hasn't been done the last days. It does say 7 users in Brazil though.
ad 6) also makes sense. As you might have noticed then I did the at central.swerts-knudsen.dk
Thanks,
Jesper
-
Hi Knuddi
Thanks about the info. Just resend the e-mail to you.
about to get two e-mails... the numbers on e-mail are different. One say I have 10 other 250 spams on junkmail (remember... in debug mode!)
If I understand the admin would receive an e-mail with report of ALL junkmail content of all users. That´s correct ?
I saw the 7 users in Brazil... I just think I have 7 (or more) servers in Brazil... for a long time.
Thanks
Jáder
-
Admin will receive an email per user just as this user would receive his/her weekly email. So in debug mode you should get multiple emails for those users that have recent junkmail. That is likely to be 10 for one person and 250 for another (unlucky soul).
The summary email should state for which user it is reporting in the "Junk emails blocked for <user> : <number>
-
Ok ... thank you...not sure how I do not see it! Maybe because one unlucky soul have 1505 messages and another one has 3066!
When running on debug mode I´m seeing this error message for some users:
Use of uninitialized value in substitution (s///) at /usr/local/unjunkmgr/spamreminder.pl line 152, <ORIGINAL> line 33.
-
Augchh - that is a lot of spam for one week.
I'll get the regex fixed - it happens if an email does not have a subject line...
-
Hi Jesper
I've installed and then uninstalled your contrib.. few minutes after "sme-unjunkmgr" I've got this mail:
/bin/bash: /usr/local/unjunkmgr/spamchanger.pl: No such file or directory
So I've checked /etc/crontab and lines of your contrib are still there.. a simple
expand-template /etc/crontab
solve the problem
HTH
Ciao
Stefano
-
For a clean un-install, you would properly also need to expand httpd.conf. I will get this into the RPM uninstall section in next release.
expand-template /etc/crontab
expand-template /etc/httpd/conf/httpd.conf
/etc/rc.d/init.d/httpd-e-smith restart
Thanks,
Jesper
-
Another question:
Why do the numbers in "Scanned for Virus" and "Scanned for spam" differ so much???
-
I think its because the two tests are made sequential (spam first I think) so only non-spam emails are AV scanned.
-
Just wanted to make sure that you had seen the updated version that has fixed the regex issue as well as doing proper cleanup in case you install..
All future documentation updates will be on: http://wiki.contribs.org/Sme-unjunkmgr