Topic: Squidgard blacklists update

[steve288]

I have attempted to read what I can on updating squidguard blacklists to current versions, regularly and  automatically but the different things I try  seem to confuse me and not work.

I have several questions that perhaps someone could answer.
First I suppose I should ask  is the ftp site that is listed in the squidguard panel current. That is is it being updated? Maybe I should just leave it alone and thats it? Currently I have...

ftp://ftp.teledanmark.no/pub/www/proxy/squidGuard/contrib/blacklists.tar.gz

Secondly I found a message (http://forums.contribs.org/index.php?topic=36217.0)
 on how to update the blacklists to shallalist.tar.gz

Basically you type the below command.

config setprop squidguard Blacklist http://squidguard.shalla.de/Downloads/shallalist.tar.gz.

I did this and it seems to recieve the command eg there are no errors but when I go to the panel and click on content filtering then look at the string in the "blacklist controls" the string there is the same as the default install. eg. ftp://ftp.teledanmark.no/pub/www/proxy/squidGuard/contrib/blacklists.tar.gz

Should this change ?

I have read several posts on trying to upgrade the blacklists but they all seem to run me into trouble.

Thanks.

 

[janet]

steve288

config setprop squidguard Blacklist http://squidguard.shalla.de/Downloads/shallalist.tar.gz.

Did you literally type that whole command as shown ?
There is an unwanted dot at the end, it should only be

config setprop squidguard Blacklist http://squidguard.shalla.de/Downloads/shallalist.tar.gz


What version are you running ?
Do
rpm -q smeserver-squidguard

It should show the latest, which is
smeserver-squidguard-1.0-22.noarch.rpm


Also what is the output of
config show squidguard

[chris burnat]

Moving to Contribs section.

[steve288]

I cannot say if I typed the . at the end of the command so I have now re done it without the dot. Most likely I just cut and pasted from the original post so as to be accurate.  I don’t know if I put the . in. Anyway I re ran the command w/out the . and again got no response, which means I assume that it ran. E.g. it didn’t have an error saying invalid command or the like.


Unfortunately the command

rpm -q smeserver-squidguard
Does not give any output?

HOWEVER this command give some output.

[root@mail2 ~]# rpm -qa |grep -i squidguard
e-smith-squidGuard-0.2-3
e-smith-squidGuard-dbgambling-010313-1
squidGuard-1.1.4-2
e-smith-squidGuard-dbwarez-010313-1
e-smith-squidGuard-dbdrugs-010313-1
e-smith-squidGuard-dbhacking-010313-1
e-smith-squidGuard-dbviolence-010313-1
warning: only V3 signatures can be verified, skipping V4 signature
e-smith-squidGuard-dbaggressive-010313-1
e-smith-squidGuard-dbporn-010313-1

The command :

config show squidguard

Gives me no feedback nothing??

So I’m confused. In the past I tried following the wiki on squidguard but it just ran me into problems and they suggest installing
http://forums.contribs.org/index.php?topic=34337.msg151745#msg151745 squidGuard 3.2 ??

It appears however I have squidguard 1.1.4-2 ?

But not smeserver-squidguard .

So now I very confused.

Also I manually added some sites to trust and block in /usr/local/squidGuard/db/trusted/domains and  /usr/local/squidGuard/db/untrusted/domains and they DO get blocked but when I click on the SquidGuard filter Panel they do not show up in the list when I click view, in fact when ever I open the files from there. When I click view or submit data I get the following message.
You didn't enter ANY data in the Domain, Expression or URL field!

Please go back and try again.
Press the back button on your browser to return to the Interface


Regards

[janet]

steve288

You appear not to have the correct version of squidguard installed ie smeserver-squidguard.
The db command will not work in that case

I would suggest you uninstall all those squidguard rpms

rpm -e e-smith-squidGuard

etc etc for all the packages you listed, you may have to sort out the uninstall order so that you don't get dependency errors

config show squidguard
Gives me no feedback nothing??

Because you have the wrong rpms installed, that do not support db commands

After removing all the squidguard packges, to be sure to reset everything to standard, do
signal-event post-upgrade
reboot

Then download smeserver-squidguard-1.0-22.noarch.rpm from
http://smemirror.fullnet.co.uk/contribs/jbennett/sme7/squidguard/RPMS/

cd to the folder on your sme server where smeserver-squidguard is located and do

yum localinstall smeserver-squidguard*.rpm --enablerepo=dag

That should install the correct package and any dependency rpms it requires. You must have the dag repository configured on your sme server (with status=disabled).

Then use the DB command to configure the blacklist and the server manager panel to control it all

Then follow the advice of mrjhb3 here: http://forums.contribs.org/index.php?topic=36217.msg160088#msg160088

which says

What on the Blacklist controls help page, do you not understand.  The blacklist setting is now a db value.  Log on to the console and do - config show squidguard.  One of the db values is Blacklist.  To change - config setprop squidguard Blacklist http://squidguard.shalla.de/Downloads/shallalist.tar.gz.
Everything that I have changed is documented in the changelog.  rpm -q smeserver-squidguard --changelog | more

If you have the latest version of the smeserver-squidguard rpm (-12), then when you download from shalla, the supdate script will now convert the lists to *.db files which load a heck of a lot faster when stopping and starting squidguard.

As far as adding entries, use the content filtering panel in server-manager.  If you have 7.1.2 or greater installed then you have to put up with the double login until I find some good time to update it.  You have trusted and untrusted (domains, urls, expressions).  Add your entries, then restart squid.  /etc/rc7.d/S90squid stop ; /etc/rc7.d/S90squid start.  You will know when things are ready when you see this in the /var/log/squidguard/squidguard.log file:
2007-03-18 17:14:54 [3687] squidGuard 1.2.0 started (1174256093.054)
2007-03-18 17:14:54 [3687] squidGuard ready for requests (1174256094.968)

[steve288]

Thanks, I think I understand.

As a general comment (not to you, but just thinking out loud) I'm not sure what the rules are for the wiki but it would be good I think if it had a clearer list like you have provided of how to install the squidguard on the server and where to get a valid software for it. I'm probably a dolt, but to me wiki's should be clear listed instructions, something like what you have done, rather than a link to a discussion where i get confused by the discussion. But then I really don't understand the process.

Thanks. I will try this out on monday.

[janet]

steve288

As a general comment....it would be good I think if it had a clearer list like you have provided of how to install the squidguard on the server and where to get a valid software for it.

Keep in mind that nobody is paid to add or update wiki articles, so if a less good wiki article exists, then it is up to someone (who choses to volunteer) to create or update the article.

Many people have created many articles, but some more work is needed.
The whole contribs.org system is a work in progress.
Obviously the wiki article for squidguard needed serious updating.

[steve288]

I have had a great deal of success in re installing the right rpm smeserver-squidguard-1.0-22.noarch.rpm.
It has taken me 4 or 5 hours to get this point.

Now when I type
[root@mail2 untrusted]# config show squidguard

I get the Happy response …
    squidguard=service
    Blacklist=http://squidguard.shalla.de/Downloads/shallalist.tar.gz
    Squidguard_Allow=trusted
    Squidguard_Block=adult,aggressive,drugs,gambling,hacking,porn,proxy,untrusted,violence,warez
    Squidguard_FullAccess=all
    Squidguard_NoAccess=none

Great thanks for your help. Everything I think seems good there.


However there are at least 3 remaining questions / Issues I don’t understand.

When I click on the panel Content Filtering /  SquidGuard Logs / View Log

I get the top line saying :

Current "SquidGuard" Log (Last 50 lines):

And then nothing listed in the log. It does not apper to be reading the squidguard.log file. If I go to the log on a terminal its fine and I can see the list of activity.
E.G.  /var/log/squidguard/squidguard.log. There is lots of activity there.

SECONDLY
When I click on the Content Filtering Panel / Blacklists Controls/Submit Data the output is as follows:

The blacklist setting is now a DB value - config show squidguard

The default squidguard blacklist is:
http://ftp.ost.eltele.no/pub/www/proxy/squidGuard/contrib/blacklists.tar.gz
which doesn't seem to be being updated.  There is another list that is being maintained,
here, http://squidguard.shalla.de/Downloads/shallalist.tar.gz, but you must adhere to their
Copyright terms if you want to use it in a commercial setting.
You can get more details here, squidguard.org

I have gone to the squidguard.org and Shalla.de but cant seem to see any information on this. I don’t know if the blacklist is updated or what exactly this means. My next step is to explore having it download automatically on a cron job or something, but I don’t know what this means.


THIRDLY and perhaps unimportant but on the link at
http://forums.contribs.org/index.php?topic=36217.msg160088#msg160088
They mention the command
db squidguard show

When I do this I get what appears to be bad data.

[root@mail2 squidguard]# db squidguard show
Database found in old location /home/e-smith/squidguard at /usr/lib/perl5/site_perl/esmith/db.pm line 714.
*WARNING* esmith::config(/home/e-smith/squidguard) called with old database path. The following package needs to be updated:  at /usr/lib/perl5/site_perl/esmith/config.pm line 376
        esmith::config::TIEHASH('esmith::config', '/home/e-smith/squidguard') called at /usr/lib/perl5/site_perl/esmith/db.pm line 545
        esmith::db::open('esmith::db', 'squidguard') called at /sbin/e-smith/db line 144

I don’t know if I should ignor this or if some file is looking in the wrong place for something or if something needs updating??


Regards

[mrjhb3]

However there are at least 3 remaining questions / Issues I don’t understand.

When I click on the panel Content Filtering /  SquidGuard Logs / View Log

I get the top line saying :

Current "SquidGuard" Log (Last 50 lines):

And then nothing listed in the log. It does not apper to be reading the squidguard.log file. If I go to the log on a terminal its fine and I can see the list of activity.

SECONDLY
When I click on the Content Filtering Panel / Blacklists Controls/Submit Data the output is as follows:

The blacklist setting is now a DB value - config show squidguard


I have gone to the squidguard.org and Shalla.de but cant seem to see any information on this. I don’t know if the blacklist is updated or what exactly this means. My next step is to explore having it download automatically on a cron job or something, but I don’t know what this means.


THIRDLY and perhaps unimportant but on the link at
http://forums.contribs.org/index.php?topic=36217.msg160088#msg160088
They mention the command
db squidguard show

The first one is a bug that I have never fixed.

The second one is informational only.  You can no longer change the blacklist setting from the panel.  Go to squidguard.org, click blacklists, then click shalla.  You should find all the info you want.  If you have changed the blacklist to shalla, then the cron file in /etc/cron.daily called supdate.cron should run and produce a report for you in the daily email that the admin gets.  Notice that the download file keeps growing.

Third it looks like you may have an old config file in the old db location.  It should be safe to remove it, or at least rename it and do you commands again.  A db squidguard show gives me nothing, but a config show squidguard gives me all.

John

[steve288]

First Id like to say thanks John (and perhaps others) for putting the work into what you have done.

On my first question, thanks, I will not worry about it.

On the second of my questions.
I looked at the cron.daily and then the supdate.cron and they all seem to be there. You mention  “Notice that the download file keeps growing.” Which file keeps growing? I’m thinking it’s the shallalist.tar.gz file. But when I do a locate shallalist.tar.gz it finds nothing. Is this the file that is suppose to grow? Since I cant even find the shallalist.tar.gz. it seems unlikely.
The report via email says the following:
/etc/cron.daily/01-rkhunter:

Warning: The following processes are using deleted files:
         Process: /usr/local/bin/squidGuard    PID: 13393    File: /var/tmp/013393
         Process: /usr/local/bin/squidGuard    PID: 13394    File: /var/tmp/013394
         Process: /usr/local/bin/squidGuard    PID: 13395    File: /var/tmp/013395
         Process: /usr/local/bin/squidGuard    PID: 13396    File: /var/tmp/013396
         Process: /usr/local/bin/squidGuard    PID: 13397    File: /var/tmp/013397
Warning: The SSH and rkhunter configuration options should be the same:
         SSH configuration option 'PermitRootLogin': yes
         Rkhunter configuration option 'ALLOW_SSH_ROOT_USER': no
Warning: Hidden file found: /usr/.directory: ASCII text

One or more warnings have been found while checking the system.
Please check the log file (/var/log/rkhunter.log)
/etc/cron.daily/supdate.cron:

04:03:06 URL: ftp://ftp.teledanmark.no/pub/www/proxy/squidGuard/contrib/blacklists.tar.gz [335999] -> "blacklists.tar.gz" [1]
Not quite sure what that means. It appears that squidGuard is using files that don’t exist? As well there is mention of the supdate.cron file and finally the ftp blacklists, with what appears to be no warning at all ?

By the way the I think the shalla site has changed a bit it says it’s a new site. The information is fairly slim. Of course it free so that is understandable under the circumstances.  I have gone over all of the links, perhaps my searching capabilities need beefing up, but for me it did not provide a lot of guidance.


On the third question:
I have read over the error very carefully, and looked at each one of the files it refers to but I’m sorry I really don’t know what script or config file to delete?
Database found in old location /home/e-smith/squidguard at /usr/lib/perl5/site_perl/esmith/db.pm line 714.
*WARNING* esmith::config(/home/e-smith/squidguard) called with old database path. The following package needs to be updat                  ed:  at /usr/lib/perl5/site_perl/esmith/config.pm line 376
        esmith::config::TIEHASH('esmith::config', '/home/e-smith/squidguard') called at /usr/lib/perl5/site_perl/esmith/d                  b.pm line 545
        esmith::db::open('esmith::db', 'squidguard') called at /sbin/e-smith/db line 144

Then when you say re run the command do you mean the
config setprop squidguard Blacklist http://squidguard.shalla.de/Downloads/shallalist.tar.gz.
command?

I have got this far I don’t want to go off half cocked and screw up my system again.

Regards

[mrjhb3]

Yes, it is the shallalist.tar.gz.  This line tells you the results:
etc/cron.daily/supdate.cron:

04:03:06 URL: ftp://ftp.teledanmark.no/pub/www/proxy/squidGuard/contrib/blacklists.tar.gz [335999] -> "blacklists.tar.gz" [1]

Notice the 335999, that's how big the file was.  This is a tar file, so the cron job untars it, does it's magic and deletes the tar file, which is why you can't find it.  If you want to watch it run do this - tail -f /var/log/messages - from the console or an ssh sesstion, then - /etc/cron.daily/suupdate.cron - from another session.  Now watch your messages log.

Can't help you with the deleted files.  Are you using some web app that caches files to /var/tmp?  Same for /usr/.directory.  I don't have this file on my server, you might want to check it out.

The file to possibly remove is /home/e-smith/squidguard.  That file isn't present on my system and it could have been the old directory where the db files were located.  First, rename it, then db show squidguard should give you nothing, then config show squidguard should show you the squidguard db settings.

John

[steve288]

John & (Mary previously)

Thanks. Things seem to be working better now. I renamed the folder you suggested and now the db squidguard show gives me nothing. I tried the test using the tail -f command. And this failed at first but it appeared there was something wrong with the network connection. Once that was up it seemed to run nicely and there was lots of activity in the log when it installed the new blacklists.
We will see how things run for the next little while. I'm setting this up for a camp for kids from the city to use the internet via windows boxes, using this as the gateway/firewall/filter.
 
Thanks again.