Koozali.org: home of the SME Server

Obsolete Releases => SME Server 7.x => Topic started by: piyushjani on October 12, 2006, 09:31:26 AM

Title: Image base Spam emails
Post by: piyushjani on October 12, 2006, 09:31:26 AM: Hi,
We are facing a major problem of receiving spam emails :( . We are using spamassassin as spam filter, but my problem is that when a spam message comes with image file attached or embedded into it , spamassassin does not detect it & pass on to user. :cry:
Can SME 7.0 help me taking care of these type of problems :lol:

Thanks in advance
Piyush Jani
Title: Image base Spam emails
Post by: mmccarn on October 12, 2006, 01:22:57 PM: I get good results with the procedure outlined in this post: http://forums.contribs.org/index.php?topic=33824.0.

There is also mention in that post of ASSP - you may want to check that out, as it is multi-platform (windows, linux, (os x?)) and is designed to do what I'm forcing SME to do - act as a spam-filtering SMTP gateway.
Title: Image base Spam emails
Post by: CharlieBrady on October 13, 2006, 04:24:11 AM: Quote from: "mmccarn"

There is also mention in that post of ASSP - you may want to check that out, as it is multi-platform (windows, linux, (os x?))

I don't see any advantage with it being multi-platform, and that likely means that it is not optimised for linux.

Quote

... and is designed to do what I'm forcing SME to do - act as a spam-filtering SMTP gateway.

That's also exactly what qpsmtpd is designed to do.

I'm not making any criticism of ASSP, but I also haven't been convinced that it has any advantages. Certainly when I first looked at it it had significant problems.

Do you think ASSP should be used rather than qpsmtpd? If so, why?
Title: Image base Spam emails
Post by: piyushjani on October 13, 2006, 06:15:50 AM: Thanks for your email :)

I have been to those links of ASSP, and also tried to search for image base spam control. I didn't find it any where :?

Does any one has tried using SME for image base spam control. Because in that image words are written , which has not business meaning its all pertaining to sex , durgs & porno. :oops:

Thanks & Regards
Piyush Jani
Title: Image base Spam emails
Post by: cactus on October 13, 2006, 09:21:14 AM: Quote from: "piyushjani"
Thanks for your email :)

I have been to those links of ASSP, and also tried to search for image base spam control. I didn't find it any where :?

Does any one has tried using SME for image base spam control. Because in that image words are written , which has not business meaning its all pertaining to sex , durgs & porno. :oops:

Thanks & Regards
Piyush Jani
I use spamassassin with Bayes filtering and it fights image based spam pretty good after the usual learning curve. To speed up the learning curve I fed the ASSP spam list to the Bayes filter (I have found the link here on the forum once). I also installed the LearnAsSpam script and also modified a copy for a LearnAsHam script as some spam was falsely detected.

On top of that I have the default RBL servers enabled.

I don't have exact figures but after using it for a few months now I almost receive no spam messages anymore in my mailbox (1 per week tops, and most of the times this one is not an image based SPAM), about of a quarter of mails received on my small mailserver is SPAM of which about half is image based.

Relevant links:

ASSP spam archive: http://easynews.dl.sourceforge.net/sourceforge/assp/asspsmpl-0.1.tgz

More usefull links and information:
http://www.sonoracomm.com/index.php?option=com_content&task=view&id=49&Itemid=32
Title: Image base Spam emails
Post by: mmccarn on October 13, 2006, 12:10:14 PM: Quote from: "CharlieBrady"
Do you think ASSP should be used rather than qpsmtpd? If so, why?
I only mentioned ASSP in case piyushjani isn't already using SME -- perhaps, for him, it would be quicker or easier to install ASSP than to setup a SME server... I have no opinion on the relative merits of qpsmtpd vs. assp as I have never used assp

Quote from: "piyushjani"
tried to search for image base spam control
I second cactus's opinion. I think you'll find (if you look at it) that your "image base spam" is really html formatted email. If it is really email containing .jpg, .gif, .png or other graphic files, you could simply setup an attachment filter.

Here are 24 Hrs of stats from one of my SME 7 boxes configured as shown in my earlier post (courtesy of the mailstats script). Note that "Misc.rejected" is artificially inflated due to 5760 smokeping EchoPingSMTP probesevery (20 probes every 5 minutes)
Code: [Select]
RBL rejected : 912 ( 11.50%) Pattern filter rejected : 0 ( 0.00%) Misc.rejected : 6082 ( 76.68%) Infected by Virus : 31 ( 3.30%) Spam rejected (over reject level): 139 ( 35.55%) Spam detected (over tag level) : 391 ( 41.68%) Ham detected (under tag level) : 489 ( 52.13%) Total emails accepted : 768 ( 81.88%) -------------------- Total emails processed : 938 ( 39.08/hr)
Title: Image base Spam emails
Post by: cactus on October 13, 2006, 01:54:35 PM: Quote from: "mmccarn"
Quote from: "piyushjani"
tried to search for image base spam control
I second cactus's opinion. I think you'll find (if you look at it) that your "image base spam" is really html formatted email. If it is really email containing .jpg, .gif, .png or other graphic files, you could simply setup an attachment filter.
I ment really graphic based spam, no html formatted image-like looking e-mail. The text is really in a graphical representation and is filtered without using attachement filtering as I did not explicitely configure this and I am not aware of it being enabled by default for SME Server.
Title: Re: Image based Spam emails
Post by: cpuffalt on October 14, 2006, 10:53:48 PM: Quote from: "piyushjani"
Hi,
We are facing a major problem of receiving spam emails :( . We are using spamassassin as spam filter, but my problem is that when a spam message comes with image file attached or embedded into it , spamassassin does not detect it & pass on to user. :cry:
Can SME 7.0 help me taking care of these type of problems :lol:

Thanks in advance
Piyush Jani

I've also been suffering from a similar epidemic of image-based spam emails getting past spamassassin. Someone suggested enabling bayesian filtering and I've had it enabled for some time but due to the random text these spams contain it's ineffective.

One possible solution might be the FuzzyOcrPlugin (http://wiki.apache.org/spamassassin/FuzzyOcrPlugin). Has anyone successfully installed this on SME server?

Corey
Title: Image base Spam emails
Post by: gregswallow on October 15, 2006, 05:48:30 AM: > One possible solution might be the FuzzyOcrPlugin

I google'd and found like what looks to be some Fedora 5 SRPMS for gocr and FuzzyOcr here:
http://mirrors.redwire.net/pub/local-rpms/SRPMS/
(I am guessing because there is a requires: giflib-utils, and that rpm is only in FC5 I think - The same thing is called libungif-progs in CentOS.)
They should be able to be modified a bit and rebuilt for SME7.

Also, the latest version of the FuzzyOcr source code can be found here:
http://www.joval.info/proj/FuzzyOcr.html
(not linked on the Spamasassin page - note that the latest FuzzyOcr version requires spamasasssin 3.1.4 or later, which SME7 doesn't have yet.)

You should add that as a NFR in the bug tracker and mention all this info. Looks interesting.
Title: Image base Spam emails
Post by: stephen noble on October 16, 2006, 10:50:29 AM: I'm using maildrop (or procmail) to sort any that gets through spamassassin to junkmail

the following sets up a global rule for all

note the quote marks around the db entry with spaces
and the back slash to escape the forward slash

I also have a rule for
'Content-Type: multipart\/mixed'

# db processmail set 41 pmGlobalRule deliver junkmail criterion 'Content-Type: multipart\/related' basis headers action sort copy no

# db processmail show 41
41=pmGlobalRule
action=sort
basis=headers
criterion=Content-Type: multipart\/related
deliver=junkmail
copy no

# signal-event mailsorting-conf

http://www.dungog.net/sme/usermanager.php#proc
Title: Image base Spam emails
Post by: gregswallow on October 17, 2006, 07:59:16 PM: > You should add that as a NFR in the bug tracker and mention all this info. Looks interesting.

I made some rpms for FuzzyOcr if anyone wants to try them - bug report is here:
http://bugs.contribs.org/show_bug.cgi?id=1985
Feedback on the bug report please.
Title: Image base Spam emails
Post by: cpuffalt on October 22, 2006, 07:31:39 AM: Greg,

Thanks for putting those rpms together. I've installed them on my home server and will let you know how they work...

I've already been noticing spam mails containing animated gifs and garbage in the background so it looks like spammers are already trying to outsmart ocr...we'll see.

Regards,
Corey
Title: Image base Spam emails
Post by: william_syd on October 22, 2006, 02:45:36 PM: An interesting read.....

http://www.secureworks.com/analysis/spamthru/
Title: Image base Spam emails
Post by: mrjhb3 on November 27, 2006, 09:54:17 PM: Quote from: "cpuffalt"
Greg,

Thanks for putting those rpms together. I've installed them on my home server and will let you know how they work...

I've already been noticing spam mails containing animated gifs and garbage in the background so it looks like spammers are already trying to outsmart ocr...we'll see.

Regards,
Corey

Corey,

How has this been working?

John
Title: Image base Spam emails
Post by: gregswallow on November 27, 2006, 10:12:38 PM: Shad has updated the rpm - try this one (but check that directory for a newer version too):
http://mirror.contribs.org/smeserver/releases/7/builds/rpms/RPMS/noarch/FuzzyOcr-3.4.2-1.noarch.rpm

plus..
http://mirror.contribs.org/smeserver/releases/7/builds/rpms/RPMS/i386/gocr-0.41-3.i386.rpm
and Spamassassin 3.1.7 from atrpms is required, and the perl modules required are from dag (rpmforge).

Feedback on the new rpm to the bug report as well please:
http://bugs.contribs.org/show_bug.cgi?id=1985
Title: Image base Spam emails
Post by: mercyh on November 27, 2006, 10:30:26 PM: Hi,

I have been watching this over on the contribs forum. Do you feel like this is ready for production?

Over the weekend with Spamassin RBLS enabled and bayese trained for 6 months I received 8 image based spam in my inbox. Another 12 hit my junkmail folder with scores between 5 and 10. I have about 60 users that don't even know what spamassin is so would like to be sure it is working before I implement.

Thanks for all your work.

Royce
Title: Image base Spam emails
Post by: gregswallow on November 27, 2006, 10:57:55 PM: > Do you feel like this is ready for production?

If it was, it would be in the smeupdates or smeupdates-testing repository.

It will be ready sooner if people test it and report bugs. Upstream has a bug tracker now as well at http://fuzzyocr.org.
Title: Image base Spam emails
Post by: cpuffalt on November 29, 2006, 07:10:13 AM: John...

Quote from: "mrjhb3"

Corey,

How has this been working?

John

Well it hasn't blown up on me. I did a quick grep through my junkmail folder and it scored a hit on about 5% of the emails there. Of course that doesn't mean spamassassin wouldn't have caught many of those without FuzzyOCR. It seems to have helped the epidemic though it hasn't cured the problem as a number of image-based spams are still getting through.

Keep in mind though that I'm just using this on my own personal server so the email volumes aren't very high. I imagine FuzzyOCR would add considerable load to a busy site.

I also have some issues with the current design of FuzzyOCR. It relies on a fixed list of keywords. It's too bad it doesn't/can't(?) leverage the existing bayesian-based scoring built into spamassassin so it could be more adaptive.

Anyhow, it's not a silver bullet but if you're running a low volume site or have a server with lots of headroom it may be worth installing... At least until a better solution appears.

Corey