Topic: Filtering attachments HELP PLEASE!!!!

[webster]

I have a little bit of a tricky question .

Currently I have a v5.5 server, its running messagewall, spamassassin and rav antivirus(which is about to come to the end of its reg period)

ATM Messagewall is only being used to reject attachments and also uses the clam virus database and it does this well to a point and has been one one of the main parts of virus protection...while looking for a (free) replacement for rav.
The reason I like messagewall was because it offers  the ability to give different levels of filtering....so 1 user can have exe's where all the others cant. This is most important for our company.
It works fine until the point that anything that comes back in from mailer-daemon doesn't get filtered...
so when you start getting hit with some viruses the come in get rejected... the other server bounces them they come back in from the  mailer-daemon then get delivered into their box (rav has been catching them ATM) its isn't good because it creates 3 times the traffic and the virus is till getting through

I've been using SpamAssassin form 3 months and extremely happy there.

Rav has been great but is  no longer an option.

I am about replace the machine with a new one and upgrade to V6 (without doing a system restore)...  we have more the 370 users so the shift  its self is a major job as it needs as little down time as possible, so i have to sort out the imap folder issue...(we have 500MB of Mail tgz to move over
I was looking at using Clam Antivirus with Amavis but our  main concern is the amount of time between the virus being release and the database updates for  Clam and  Amavis.

I have looked at assp but that only allows global filters from what i have seen...

(i have noticed the bad-attach  by Abe Loveless http://www.tech-geeks.org/contrib/loveless/bad-attach/README.txt this looks hopfull but doesn't work on v6 yet AFAIK)

Has anyone got any solutions that will allow me to filter certain attachments on a per user basis? I would be extremely grateful and welcome any suggestions... I only really have a month window

Thanks
T

[PeterG]

Blimey.

Can't help with the virus stuff as I have all that outsourced, much easier.

With the server migration though, are you using the server as a domain controller too? If you are, then you might need more than a month...


PeterG.

[webster]

nope... its only purpose is a mail server... i actually copied (parts of) the /etc/shadow, /etc/passwd, /etc/groups
files as i don't have a record of all the users passwords
and /home/e-smith/accounts
over to the new server that all works fine... its the stored mail that is the issue ... i tried
cd /home/e-smith/files/users/
tar zcvf SomeUser.tgz SomeUser/

then extracted it on the new box...this was a 25mb tgz file and
after extracting it didn't work but i ran the /sbin/e-smith/signal-event post-upgrade after a few mins i was able to restart and that worked fine.....

so i tgzed the whole user dir...taking 30 mins  or so  now with a 500mb tgz so i deleted the users dir and extracted the new one ... only took 5 mins to extract
tried the post-upgrade
it takes over 5 or more hours and still don't pick up the old imap folders

anyone got any ideas as a side issue to my mail filtering issue?

[Pete]

I ahve a pair of MailSacanner servers running on RH9 that filter all the mail for all of our domains/email networks - clamav is the av tool on both and i dindt have to lift a finger today as those servers got punished by this virus outbreak. Clamav updates hourly and was already up todate and trapping attachments. We have 3 levels of AV below this one, but clamav is on the border machines, then SAV on mailservers, NAV on fnp, and etrust on workstations.

Seriously consider MailScanner - it handles your levels of scanning etc. Also have a look at using some custom rule sets in spamassain that will assist you trapping the crappy attachment - bigevil.cf is a beauty. There is a mailscanner contrib available - might even be worth having seperate box to run this, as mailscanning can get busy, eps if you hav web users trying to lod pages etc.

I run MS on a p200 NEC server handles around 2500 messages per day without raising a sweat.

MailScanner has GREAT support, VERy easy to set up and maintain, there is a contrib and there are some 3rd party UI bits for it, but hardly needed.

[webster]

Seriously consider MailScanner - it handles your levels of scanning etc. Also have a look at using some custom rule sets in spamassain that will assist you trapping the crappy attachment - bigevil.cf is a beauty. There is a mailscanner contrib available - might even be worth having seperate box to run this, as mailscanning can get busy, eps if you hav web users trying to lod pages etc.

is that dungogs MailScanner ??? if not care to point me in the right direction save hunting round for it?

currently our machine eats it
its a p4 1.6
but i am actually replacing it with a p4 2.8... couldnt get anything less from out supplier!!!! i know its overkill but it does alot of mail...we will soon have 500+ users

just as an idea of how many mails we get for the 370

Completed messages: 39517
Recipients for completed messages: 41093
Bytes in completed messages: 581040320
Bytes weighted by success: 584870393
Time span (days): 2.15854

so we do almost 10 times what you do

T

[Mumm-Ra]

ASSP does allow individual filtering but requires a config change.  The following page explain how.
Filtering attachments
http://assp.sourceforge.net/fom/cache/37.html

[Derek L]

Try this for your IMAP migration:

http://home.arcor.de/armin.diehl/imapcopy/imapcopy.html

It worked for me with approx 50 users going from 5.1.2 to 6.0

[loveless]

cd /home/e-smith/files/users/
tar zcvf SomeUser.tgz SomeUser/

then extracted it on the new box...this was a 25mb tgz file and
after extracting it didn't work but i ran the /sbin/e-smith/signal-event post-upgrade after a few mins i was able to restart and that worked fine.....
[\quote]

Really??  Is the new server setup exactly the same as the old one?  Hostname and Domain name specifically?  When I tried this, my servers hostnames were different.  So, the names of the existing mail messages didn't correspond, and weren't viewable by the IMP.  As I understand it, incoming messages get tagged with an ID.HOSTNAME.  My new server couldn't read the messages ending with the old HOSTNAME.


This thought just came to me... what about doing an rsync from one server to the other?  That would copy all the data from one to the other... not sure if it would keep permissions the same or not, but a post-upgrade event would refresh any confused permissions.

There's a section in this dealing with rsync over SSH, which would probably be the easiest.
http://www.tech-geeks.org/contrib/loveless/rsync_backup/SME_BackupServer.html

As for my bad-attach contrib.  Yes, it's still in the works... but essentially what it does is use procmail to filter the messages.  Messages can either be moved to a "bad-attach" IMAP directory, or be dropped altogether.  I'm still toying with making it a little more useable... adding variables to do this or that.

I think the fix is a quickie for moving it to 6.0.  I'll try to make some progress on it tonight.

[loveless]

Ok, my bad-attach contrib has tested correctly on my SMEServer v6.0 test box.

http://www.tech-geeks.org/contrib/loveless/bad-attach/

Get the newer one.  

I also give you a few more options than were previously available.  Now you can:

1.  Choose to forward a notice to admin (only message headers, now)
2.  Choose to send a notice to recipient, or not
3.  Edit the message that goes to the recipient
4.  Choose to move bad message to "bad-attach" directory, or just drop it.
5.  Choose SME Server v5.x or 6.x
6.  Modify the file types to be blocked.

Let me know how you get along.


Note:  I've been running a variation of this since late last week.  I've been seeing a lot of messages that are getting blocked that maybe shouldn't  For instance, it blocks a lot of mailings from Ebay because it detects a .com attachment... but I haven't been able to find the attachment.  The messages do have a ton of html code in them, so who knows what they're actually doing.  I'll keep looking for a way to de-sensitize it, though.

[webster]

Really??  Is the new server setup exactly the same as the old one?  Hostname and Domain name specifically?  When I tried this, my servers hostnames were different.  So, the names of the existing mail messages didn't correspond, and weren't viewable by the IMP.  As I understand it, incoming messages get tagged with an ID.HOSTNAME.  My new server couldn't read the messages ending with the old HOSTNAME.

Oops sorry i should have added how i transfered my mail (and users) across

its long and i hope its not to hard to understand

step 1, download and install lazy admintools
http://www.ibiblio.org/contribs/contribs/mblotwijk/Contribs/lazy-admin-tools/

step 2 run the lat-dump and copy the outputted files accross to your new server that has been freshly installed.

step3 on the new server install the lazy admin tools
then run lat-restore
this will set up all your users ibays procmail, groups uids etc...everything really!!! just not the passwords!

step4
you will need to either reset the users passwords to new ones (you could at them to your (servername).Users

if you don't know the passwords the other options is reset them all the users passwords in te server-manager to something simple (copy & paste).... so the /home/e-smith/accounts registers that your users have a password. then
you can copy the shadow password file across from your old server to a temp directory...now you need to be real careful ppl... or you will have to reinstall you server... delete the newly created users from the /etc/shadow file
edit your old shadow file in your temp dir to only have your off the other machine (not root admin qmail www etc) now you can "cat tempshadowfile >> /etc/shadow"
that transfers your old passwords over to the end of the new servers shadow file
(note that if you  don't reset the passwords in the server manager first...the server will reset all new users passwords at next/every "post-upgrade"!!!!)

step5
use webmail to test a user you know the password of and make sure it is working.

step6
now... for the mail... i edited the backup restore script (/etc/e-smith/web/functions/backup)... i made it only back up /home/e-smith/files/users
setp7
then  run the backup to my desktop from the server manager then restored that to the new server... once i did that it said it had an error and didn't complete.ignote it

step8 now run /sbin/e-smith/signal-event post-upgrade
/sbin/e-smith/signal-event reboot
then it should all work !!!!

step9
if it does work now is a good time to do a backup on the new server

thats a simple description... but give you a basic idea on how to do it ... if you need any help lemme know...may not be the best or safest way of doing it so don't complain...but it worked so i am happy

[Michiel]

the other options is reset them all the users passwords in te server-manager to something simple (copy & paste).... so the /home/e-smith/accounts registers that your users have a password.

Or even better: After you run lat-dump, edit the lat-restore file to read lat-users -p -a -i=(yourserver).Users.  The -p switch will generate random passwords for each account and write them to passwords.new. Either give each user his/her new password, or continue as Webster described in the second half of step 4

you can copy the shadow password file across from your old server to a temp directory...now you need to be real careful ppl... or you will have to reinstall you server... delete the newly created users from the /etc/shadow file edit your old shadow file

Hmm, it shouldn't be too difficult to write a script for that. I'll put it on my 2do list. Would be a nice addition to the lazy admintools.

Michiel

[webster]

the other options is reset them all the users passwords in te server-manager to something simple (copy & paste).... so the /home/e-smith/accounts registers that your users have a password.

Or even better: After you run lat-dump, edit the lat-restore file to read lat-users -p -a -i=(yourserver).Users.  The -p switch will generate random passwords for each account and write them to passwords.new. Either give each user his/her new password, or continue as Webster described in the second half of step 4
Michiel

Very true!!... i was in such a hurry to get it done i wasn't wanting to try my luck too much!

you can copy the shadow password file across from your old server to a temp directory...now you need to be real careful ppl... or you will have to reinstall you server... delete the newly created users from the /etc/shadow file edit your old shadow file

Hmm, it shouldn't be too difficult to write a script for that. I'll put it on my 2do list. Would be a nice addition to the lazy admintools.

Michiel

So cool! you rock Michiel, LAT was such a time saver! it really saved the day, I am so glad i didn't have to put them back in ny hand!... thankyou sooooo much