Topic: [ANNOUNCE] new contrib for web filtering

[Daniel B.]

Hi.

I just want to annonce the availability of a new contrib which adds web filtering capabilities. It uses squidguard to block URLs and squidclamav to scan content preventing viruses. There's a simple panel for the server-manager to configure everything. The contrib also logs every requests in a MySQL database for easy analysis of the traffic.

This contrib was made to be simple to configure. It's not as powerful as dansguardian can be, but is really easier to install and is more user friendly.

You can take a look at the wiki page here: http://wiki.contribs.org/WebFiltering

Regards, Daniel

[chris burnat]

Thanks Daniel, sounds great!
Some questions if I may:
- How good is the database for blocked sites, and how to check if and when it was last updated?
- Should Dansguardian be uninstalled before installing your contrib, or will disabling Dans be OK to test?
Thanks.

[Daniel B.]

Some questions if I may:

Of course you may

- How good is the database for blocked sites, and how to check if and when it was last updated?

Databases are quite good, at least for my needs. The adult one (which list porn websites) is the most complete.  If you want to see which database was modified, you can edit /etc/cron.daily/squidGuard and remove 2>&1 > /dev/null at the end. You'll receive an email each night where you'll be able to see which database have changed. You can also disable the autoupdate feature and manage your own list, you can find several databases in squidGuard format on the internet.

- Should Dansguardian be uninstalled before installing your contrib, or will disabling Dans be OK to test?

In theory, it should work when dansguardian is installed. I haven't tested thought (I've developed this because I find dansguardian really too much a pain to install and manage).

Regards, Daniel

[chris burnat]

(I've developed this because I find dansguardian really too much a pain to install and manage).
Regards, Daniel

Same here, I use Dans a fair bit in community based organisations, and it can be a real pain....  Will test and report problem in Bugzilla if applicable.  Many thanks again. Cheers. cb

[p-jones]

[root@server2 ~]# yum --enablerepo=epel --enablerepo=fws install smeserver-webfilter
Loaded plugins: fastestmirror, protect-packages, smeserver
Error getting repository data for epel, repository not found

Is there a fix for this ??

[Daniel B.]

Yep, as explained in the wiki page, you need to configure the EPEL repo (and Fws too). See http://wiki.contribs.org/Epel and http://wiki.contribs.org/Fws

Regards, Daniel

[hawk]

hi
just what i have been looking for. never liked Dans and have been using untangle on a separate server for years. this contrib has no bells or whistle, easy and simple.  installed and working great.
thanks very much
john

[hawk]

hi Daniel

Thanks again this is a great contrib, works wonders for me, only had it running for 24 hours and already there are people starting to dislike me. Please can i ask a few questions?
when the blocked screen comes up there is a place for client name, user & group how do i set this part up?
how can i add information to the the blocked page?
Many thanks
John

[Daniel B.]

The username would need to enabled user authentication on squid. This is would require some custom templates to enable it, and to configure all your client to use the proxy (authentication doesn't work in transparent mode). For the group part, it's just because squidGuard support several group of people with different filtering rules. As this contrib was made to be simple, this feature is not really used (well, there's two groups in fact: the default one, which get filtered and the unrestricted one, which is not filtered at all).

The default blocked page is /usr/share/squidGuard/cgi-bin/blocked.cgi (which is available at http://hostname.domainname.com/squidGuard/cgi-bin/blocked.cgi). You can customize the URL to which blocked requests are redirected, for example, just copy /usr/share/squidGuard/cgi-bin/blocked.cgi to /usr/share/squidGuard/cgi-bin/custom.cgi, make the change you want, and set it as the default location for blocked pages:

Code: [Select]

db configuration setprop squidguard RedirectURL \
http://hostname.systemname.com/squidGuard/cgi-bin/custom.cgi?clientaddr=%a&clientname=%n&clientuser=%i&clientgroup=%s&targetgroup=%t&url=%u
signal-event http-proxy-update


[nicolatiana]

Hi Daniel
 
I'm testing too; do you plan to put the contrib in "translation" ? If so keep us informed so that I can provide the italian translation.
Many thanks for Your job.
 
Nicola
 

[chris burnat]

Hello Daniel,
Looking great hey?  You have a lot of good information in this thread, would it be possible to incorporate it in the Wikis, or if time is not available, put a link to this topic flagging additional information being available.  Working for me like a charm on my prod box, thanks for your work.

[Daniel B.]

The wiki page is now updated

[nicolatiana]

Many thanks; I'll deal with italian translation. Maybe I'll leave categories main voice <base> in English just to avoid alphabetical order issues.
If you think possible I'll put the transalation between () something like this:
 
    <entry>
        <base>adult (adulti)</base>
        <trans>Siti per adulti che contengono materiale pornografico sia erotico che hardcore</trans>
    </entry>
 
If not possible I'll leave simply in english

Nicola

[Daniel B.]

Nop, you cannot modify the word in <base>, you just have to translate the <trans> part. Categories names are not translated for now, they appear in the panel just as they appear in the file system (in /var/squidGuard/blacklists)

[cno]

Thanks for Excellent web filtering contrib

it works very fine and do the job

but I can't find how to block https sites ?? eg https:\\minecraft.net

[Daniel B.]

You cannot filter https URL, because it's encrypted, squid doesn't see the full URL. But you can deny complete domain name. Just add minecraft.net to the blacklist, and manually configure your client to use the proxy (IP of your SME Servver, port 3128). When a proxy is manually configured for HTTPS, the browser asks squid to tunnel the traffic, so, even if squid cannot see the full URL, it can see the domain name, and deny it if it's blacklisted.

Regards, Daniel

[hawk]

Hi Daniel

I have loaded this contrib on several servers and it works like a charm. Today i am loading on another and it just will not work. It doesn't block any thing. Even if i put the domain in the blacklist.

Simple example.

all the categories are ticked and i have sex.com in the blacklist, all the ip and gateway settings are correct, and still i can view the site. all the computers on the network can view the site and other sites that should be blocked.

SME server v8, fully up to date, reboot, reconfigured done twice

Do you have any idea as to what might be the problem.

thanks
John

[Daniel B.]

You should take a look at the log file /var/log/squidGuard/squidGuard.log

[chris burnat]

[snip]
Do you have any idea as to what might be the problem.

thanks
John

Please open a ticket at Bugzilla providing information about your server and including logs (attach them) as requested,  i.e. /var/log/squidGuard/squidGuard.log
Thanks.

[hawk]

hi

thanks for the reply. I looked through the logs and found line errors.

i did a total uninstall rebooted and installed again.

now all is working sweet
thanks
great contrib.

[calisun]

Thank you, this is  a great contrib and it works great.

I do have one issue, when I enable the AV Filtering, I am unable to download any apps from Apple store on my iPhone. When I disable the AV Filtering, the app download works fine.

Is there a way to change AV Filtering setting so I can have AV Filter enabled and I can download apps from the Apple Store.

[Daniel B.]

Yep, this is because when AV filtering is enabled, the AV engine rewirte the UserAgent string, and the app store will deny the connection. To fix it, you can add *.phobos.apple.com in the whitelist. You'll have the same problem wioth the android playstore, and you can fix it adding:

clients.google.com
android.clients.google.com

In the whitelist too.

Regards, Daniel

[p-jones]

I do have one issue, when I enable the AV Filtering, I am unable to download any apps from Apple store on my iPhone. When I disable the AV Filtering, the app download works fine.

Interesting Comment. I have same issue downloading for my android phone from Google PlayStore.

EDIT: Ok this has been answered . We both posted together !

[calisun]

Thank you for a super quick reply.

I did read some suggestions on other web pages to whitelist:
itunes.apple.com
ax.itunes.apple.com
albert.apple.com
gs.apple.com

But no luck.

I will also add your suggestion
*.phobos.apple.com

And see if it will fix the issue.

Thanks again.

[hawk]

Hi Daniel

Not sure if this is related, but i am getting new errors appearing on a few of my server.

1.Backup terminated: pre-backup failed - status: 256

when i check the logs i find the following.
May 19 22:27:03 blomlinuxserver esmith::event[21297]: Running event handler: /etc/e-smith/events/pre-backup/S20mysql-dump-tables
May 19 22:27:09 blomlinuxserver esmith::event[21297]: mysqldump: Got error: 144: Table './squid_log/access_log_04_2013' is marked as crashed and last (automatic?) repair failed when using LOCK TABLES 
May 19 22:27:09 blomlinuxserver esmith::event[21297]: S20mysql-dump-tables=action|Event|pre-backup|Action|S20mysql-dump-tables|Start|1368995223 986295|End|1368995229 609257|Elapsed|5.622962|Status|256
M

then on another server a simular error

2013-05-15 04:03:34 [25109] init domainlist /var/squidGuard/blacklists/shopping/domains
2013-05-15 04:03:34 [25109] loading dbfile /var/squidGuard/blacklists/shopping/domains.db
2013-05-15 04:03:34 [25109] init urllist /var/squidGuard/blacklists/shopping/urls
2013-05-15 04:03:34 [25109] loading dbfile /var/squidGuard/blacklists/shopping/urls.db
2013-05-15 04:03:34 [25109] /usr/bin/squidGuard: can't write to logfile /var/log/squidGuard/squidGuard.log
2013-05-15 04:03:34 [25109] squidGuard 1.4 started (1368583414.133)
2013-05-15 04:03:34 [25109] db update done
2013-05-15 04:03:34 [25109] squidGuard stopped (1368583414.609)
chown: cannot access `/var/log/squidGuard/*': No such file or directory
chmod: cannot access `/var/log/squidGuard/*': No such file or directory

These servers are giving different errors, but both lead back to Squid / Squid Guard / Squid log.

Just a shot in the dark do you think this could be related to the web filtering contrib?
Any ideas would be most welcome

thanks
john