Koozali.org: home of the SME Server

e-mail filtering

Bruce

e-mail filtering
« on: May 25, 2001, 09:36:17 PM »
E-mail may be the blessing of the 21st century, but junk mail is not.  While AOL may not be my ISP of choice, they have done some remarkable things with e-mail that block all messages from sites that are not registered.  It appears they check the name server from which the mail is received, and delete all messages that are NOT from registered name servers.  This would be a great capability for e-Smith.

Also, an interface to filter all e-mail by named user or server would also be useful.  While I get about 200 e-mails a day, about 175 of them could be removed using a valid set of rules.

SAVIN Jérome

Re: e-mail filtering
« Reply #1 on: June 22, 2001, 02:46:28 PM »
I would like to have an interface for this too , but you can do it from now !

It's long, but you can do it with .qmail files and condredirect rule.
You can take a look :
       less /var/qmail/man/man1/condredirect.1

Otherwise, I can give you a shell script (home made and first version, so be lenient... ) which compares $SENDER variable with a list of SPAMMERS adresses.

Contact me for more informations.

I would like to make a module for e-smith manager, but I don't khnow how !

Bye.

SAVIN Jérome

Salim Lakhani

Re: e-mail filtering
« Reply #2 on: July 03, 2001, 07:09:42 AM »
doesn't e-smith come configured with procmail?  it's pretty easy to setup "recipes" to do this.... Maybe there's a web interface to this?

Nathan Fowler

Re: e-mail filtering
« Reply #3 on: September 28, 2001, 07:00:59 AM »
I know this topic is a bit stale however I would be very curious to see the code developed by SAVIN.  If it is feasible to modify the source code in conjunction with a cron job using lynx --dump to periodically update the black-hose list this would be very useful.  I know many of us would love to have a working solution to SPAM, I grow very tired of seeing it in my Inbox.

A few questions I have is does the code use a .qmail file for each users, or a .qmail-default?  
Has the .qmail file been created?
Will this method work with E-SMith 4.1.2?

If we can get a working script, semi-bug free, with scheduled black-hole list updates I would love to see this as a standard option in future versions of 'E-Smith'.  I know anyone who runs a mail server is plagued with SPAM and powerless to stop it, unless you name is Savin of course.

Thanks in Advance,
Nathan

SAVIN Jérome

Re: e-mail filtering
« Reply #4 on: September 28, 2001, 06:34:32 PM »
Hello,

You can try with this, it works for me on e-smith 4.1.2.
Create a shell script called default_spammer_robot in /var/qmail/bin (qmail looks only in that folder). This script should be as follow :
#!/bin/bash
# SAVIN Jérome 20010620

IFS=:

SPAMMERS=toto@titi.com:titi@toto.com
for SPAMMER in $SPAMMERS;
do
    if [ "$SPAMMER" == "$SENDER"]
    then
      echo -e "Rejecting mail form $SENDER to $RECIPIENT at date" >> /var/log/rejected_mails.log
      exit 0
    fi
done
exit 1

SPAMMERS variable should contain the list of adresses from which you don't want to receive mail.

As for default_spammer_robot, you can create a specific one for each user called my_e-smith_user_spammer_robot, and the shell should be normaly the same. And the user have to configure SPAMMERS variable and eventually the log file.

Then in /home/e-smith/files/users/my_e-smith_user edit the .qmail file and add BEFORE the ./Maildir line
|condredirect $SENDER default_spammer_robot
|condredirect $SENDER my_e-smith_user_sammer_robot

Finished.

We can have eventually set the spammer list in another file.

I hope it's helpfull for someone and I apologize for my poor english.

I'm wating for your comments.

Bye.

SAVIN Jérome

Nathan

Re: e-mail filtering
« Reply #5 on: September 30, 2001, 06:37:51 AM »
I was able to create some Bash code that runs against a file, attached is the code.  This is still beta and probably inefficient and buggy.  If you can devise a better way please let me know.


[File /home/e-smith/files/users/%username%/.qmail]
|bouncesaying 'The message was not delivered because it was viewed as SPAM mail.' spamfilter $SENDER $RECIPIENT
./Maildir/
#Template End#



[ FILE /var/qmail/bin/spamfilter]
!/bin/bash
#// Nathan Fowler
#// evilghost@stickit.nu
#// Code adapted from SAVIN Jerome
#// Please visit http://forums.contribs.org/index.php?topic=19703.msg77706#msg77706 for more info.

#Location of file containing list of Spammer Addresses
#Should be in the format of one address per line
Filename=/var/spamfilter/spamlist.txt

#// Do not modify this code, doing so can break it.
if [ -z "$1" ]
then
        echo "date Sender not passed!" >> /var/spamfilter/spam_filter_errors.log
        exit 1
fi

SRCADDR=$(echo $1 | tr A-Z a-z)

if [ -f "$Filename" ]
then
        echo "Found file!" > /dev/null
else
        echo "date Cannot find spam address file! $Filename" >> /var/spamfilter/spam_filter$
        exit 1
fi

cat $Filename | tr A-Z a-z > /tmp/spamlist.txt
Filename=/tmp/spamlist.txt
counter=0
line_count=wc < $Filename | awk '{ print $1 }'
until [ "$counter" == "$line_count" ]
do
  let "counter+=1"
  read spamaddr
  if [ "$spamaddr" == "$SRCADDR" ]
  then
        echo -e "date Denied $SRCADDR to $2" >> /var/spamfilter/rejected_mails.log
        exit 0
  fi
done <"$Filename"
exit 1

Nathan

Re: e-mail filtering
« Reply #6 on: September 30, 2001, 06:39:47 AM »
Oh, I forgot to add, the above script was tested with a 35,856 line text file containing spammer addresses.  It doesn't take too long, I'd say about 5 seconds.

Quade

Re: e-mail filtering
« Reply #7 on: October 01, 2001, 06:23:23 AM »
I tried using Nathan's modified version, using the copy-n-pasting method, and filtered out one of my other e-mail accounts as 'spam.' However, instead of blocking the e-mail, I get three copies of the e-mail instead.

Any ideas?

Nathan Fowler

Re: e-mail filtering
« Reply #8 on: October 01, 2001, 08:29:04 AM »
Quade, chances are you may have a misconfigured .qmail file.  The HTML code on this forum wrapped the '|bouncesaying ....' line.  This should really be one line.  You should have in your .qmail file:

#blah blah disclaimer about E-Smith and templates
|bouncesaying .... ..... ....
./Maildir/
# End Template


Let me know if this fixes your problem.  Also, a friend and I are in the process of converting this Bash code to C to reduce over-header, increase speed, and make it nicer for the user.  When this is finished I will create documentation for installation, create an installation script, and an e-smith custom template for global distribution to users.  Until then you can use the bash code if you wish.  ETA to deployment of the C code is less than one week.

Also, make sure your spamfile.txt looks something like this:

[Contents of spamfile.txt]
spammer@spamwhore.com
annoying@annoying.com
freeporn@smutcity.com
[End File]

Like I said, I was able to get it working successfully with a 35,862 line file full of addresses with no problem, I was only disappointed by the runtime it took, I am confident that C will be our answer.  Make sure you don't have your e-mail address in the spamfile.txt, if so you'll bounce your own messages (duh).

Hope this helped, let me know.

Nathan Fowler

Nathan Fowler

Re: e-mail filtering
« Reply #9 on: October 01, 2001, 11:47:10 AM »
We currently have a working, fast, and efficient program that will parse over 35,862 addresses in under one second, literally.  The code is working great and has been converted from bash to C++, accepting several user-configurable command-line arguments.  It is necessary to install compat-libstdc++-6.2-2.9.0.14.i386.rpm if it does not exist on the system.  I am currently in the process of writing documentation for this program.  I plan on having the source code, the compiled binary, a starter spamlist.txt, and detailed installation instructions in the next two days.  Please just hang tight until then.

Nathan Fowler

Allen Rapini

Re: e-mail filtering
« Reply #10 on: October 02, 2001, 06:40:22 PM »
Nathan
I am interested in testing this application when you are ready. You can contact me on or off the forum when you think you have something to test.
I have a test machine, and I am not afraid ot use it... :>)

Allen

Nathan

Re: e-mail filtering
« Reply #11 on: October 02, 2001, 08:01:25 PM »
I've been testing it on my box and it seems to be working perfectly and flawlessly.  We are going to make some minor changes to the application before the final production version.  Allen, if you are willing to help test this 'pre-beta' version let me know, I can give shoot you a quick email with the binary and some brief directions, let me know.

NF
evilghost@stickit.nu

RIchard Emory

Re: e-mail filtering
« Reply #12 on: October 05, 2001, 10:54:18 PM »
Nathan,

Newbie question here.  If tht c++ library is installed, will this compromise the security of the system?  If so will you also publish the bash only script for the paranoid?  Either way,  great job on solving a big problem.  BTW, I would also like to test your compiled code on my home server.  If you would like to see how bad a newbie can screw up your work, just let me know.

Nathan

Re: e-mail filtering
« Reply #13 on: October 06, 2001, 03:21:25 AM »
Security issues?  None that I know of, it only provides the C++ libraries for binary execution, just like you have libc on there.

Patrick B

Re: e-mail filtering
« Reply #14 on: November 15, 2001, 05:52:13 AM »
Nathan,

How's that filtering program coming along?  I'm very interested in getting SPAM filtering in place on my server.

Just curious, and update on the status if you could - thanks.

Regards,
Patrick