Koozali.org: home of the SME Server
Obsolete Releases => SME Server 8.x => Topic started by: purvis on August 14, 2011, 11:49:49 AM
-
i have been working my tail off on squidguard for sme 8.
and i have done some learning with compiling the last stable version of squidguard 1.4 for sme 8
the version places a lot of files in directories that probably sme standardized so there is a lot of do for a newbie
has anybody done anything with the squidguard on sme8 before i pull out what hair i have left
i assume squidquard is what i am looking for or what i may have to work with in the results i want.
basically i want easy squid control and some automation.
i want to host a website where blacklist and white list will be placed, then have the sme servers in an automated fashion to download the files needed to replace what is on remote servers.
thanks in advance
-
edited august 17, 2011
i am a novice on linux and here is how i installed squidguard 1.4 on sme8beta6
these scripts inorder will
download two files off the internet
install the compiler and other needed files such as the source code
compile the source files to create the program squidguard 1.4 called "squidGuard" in a directory called "/temp"
then create new directories on the sme server
copy the program squidGuard into the /usr/bin directory
then create files for squidGuard to use
then stop and restart the squid server there by starting the squidGuard service
i broke it down in steps so it can be more easily copied
there is just one reboot in the process
it would be probably good to reboot after the end as well with "signal-event post-upgrade;signal-event reboot"
these steps where created from some hard knocks
i have some more notes at the bottom
i use putty on windows to get a linux terminal
log into the sme server with putty
and run the the below
you need to make sure you have internet access on the sme server
this first step will cause a reboot automatically
cd /
mkdir -p /temp
cd /temp
wget 'http://www.squidguard.org/Downloads/squidGuard-1.4.tar.gz'
tar -zxvf squidGuard-1.4.tar.gz
sleep 15
yum -y install db4-devel
sleep 15
yum -y install gcc
sleep 15
signal-event post-upgrade
sleep 15
signal-event reboot
give your sme server some time to stablize after booting back up
i run the command "yum clean all" and look for something like below:
Loaded plugins: fastestmirror, protect-packages, smeserver
Cleaning up Everything
after a reboot then log back into the sme server with putty
run the below
cd /
cd /temp/squidGuard-1.4
./configure --prefix=/usr/bin --with-sg-config=/etc/squid/squidguard.conf --with-sg-logdir=/var/log/squidguard --with-sg-dbhome=/var/lib/squidguard/db
sleep 5
make
ls /temp/squidGuard-1.4/src
sleep 10
cd /
cp /temp/squidGuard-1.4/src/squidGuard /usr/bin
chmod 755 /usr/bin/squidGuard
mkdir -p /var/lib/squidguard/db
mkdir -p /var/log/squidguard
rm -rf /var/log/squidguard/squidGuard.log
cp /etc/squid/squidguard.conf /etc/squid/squidguard.conf.new
rm -rf /etc/squid/squidguard.conf
chown -R root /var/lib/squidguard/*
be sure to do the code in order and do the code above before doing the code below
cd /
mkdir -p /etc/e-smith/templates-custom/etc/squid/squid.conf
cd /etc/e-smith/templates-custom/etc/squid/squid.conf
echo "" > 45squidGuard
echo "redirect_program /usr/bin/squidGuard -c /etc/squid/squidguard.conf" >> 45squidGuard
echo "redirect_children 5" >> 45squidGuard
echo "" >> 45squidGuard
sleep 3
cd /
expand-template /etc/squid/squid.conf
sleep 5
mkdir -p /var/lib/squidguard/db/dame
echo "# write domains you'd like to prohibit access to" > /var/lib/squidguard/db/dame/domains
echo "yahoo.co.jp" >> /var/lib/squidguard/db/dame/domains
echo "example.com" >> /var/lib/squidguard/db/dame/domains
mkdir -p /var/lib/squidguard/db/dame
echo "# write URLs you'd like to prohibit access to" > /var/lib/squidguard/db/dame/urls
echo "www.yahoo.co.jp/deny/" >> /var/lib/squidguard/db/dame/urls
echo "www.example.com" >> /var/lib/squidguard/db/dame/urls
chown -R root /var/lib/squidguard/db
mkdir -p /etc/e-smith/templates-custom/etc/squid/squidguard.conf
cd /etc/e-smith/templates-custom/etc/squid/squidguard.conf
echo "#" > 10abegin
echo "# CONFIG FILE FOR SQUIDGUARD" >> 10abegin
echo "#" >> 10abegin
echo "" >> 10abegin
echo "dbhome /var/lib/squidguard/db" > 10adbhome
echo "logdir /var/log/squidguard" > 10alogdir
echo "" > 20destdame-a
echo "# define a category dame" >> 20destdame-a
echo "dest dame \{" >> 20destdame-a
echo " # define prohibit domain file for dame" >> 20destdame-a
echo " domainlist dame/domains" >> 20destdame-a
echo " # define prohibit URL file for dame" >> 20destdame-a
echo " urllist dame/urls" >> 20destdame-a
echo "\}" >> 20destdame-a
echo "" >> 20destdame-a
echo "acl \{" > 20destdame-b
echo " default \{" >> 20destdame-b
echo " # allow except dame" >> 20destdame-b
echo ' pass !dame all' >> 20destdame-b
echo " # redirect URL" >> 20destdame-b
echo " redirect http://localhost" >> 20destdame-b
echo "\}" >> 20destdame-b
echo "\}" >> 20destdame-b
echo "" >> 20destdame-b
rm -rf /etc/squid/squidguard.conf
expand-template /etc/squid/squidguard.conf
sleep 10
rm -rf /var/log/squidguard/squidGuard.log
chown -R root /var/lib/squidguard/db
this step should now restart your squid service and then you should have squidguard installed
cd /
svc -d /service/squid
sleep 10
/sbin/service squid restart
at this point you should be finished installing squidguard 1.4
it might be nice for you to review the squid.conf and squidguard.conf files
for the squid.conf file you, you are looking for the two lines as below that should be included in the file
less /etc/squid/squid.conf
redirect_program /usr/bin/squidGuard -c /etc/squid/squidguard.conf
redirect_children 5
for the squidguard.conf file
less /etc/squid/squidguard.conf
#------------------------------------------------------------
# !!DO NOT MODIFY THIS FILE!!
#
# Manual changes will be lost when this file is regenerated.
#
# Please read the developer's guide, which is available
# at http://www.contribs.org/development/
#
# Copyright (C) 1999-2006 Mitel Networks Corporation
#------------------------------------------------------------
#
# CONFIG FILE FOR SQUIDGUARD
#
dbhome /var/lib/squidguard/db
logdir /var/log/squidguard
# define a category dame
dest dame {
# define prohibit domain file for dame
domainlist dame/domains
# define prohibit URL file for dame
urllist dame/urls
}
acl {
default {
# allow except dame
pass !dame all
# redirect URL
redirect http://localhost
}
}
-
removed because not needed
-
cd /temp/squidGuard-1.4
./configure --prefix=/usr/bin --with-sg-config=/etc/squid/squidguard.conf --with-sg-logdir=/var/log/squidguard --with-sg-dbhome=/var/lib/squidguard/db
make
make install
and after the compiling
mkdir -p /var/lib/squidguard/db
mkdir -p /var/log/squidguard
rm /usr/bin/bin/squidGuard
rm -rf /usr/bin/bin
rm -rf /usr/bin/squidGuard
rm -rf /var/log/squidguard/squidGuard.log
cp /temp/squidGuard-1.4/src/squidGuard /usr/bin
chmod 755 /usr/bin/squidGuard
Why are you removing stuff that is installed by the 'make install' command and replace them with other files? If you use the prefix /usr/bin/ a 'make install' would normally install the program there with the proper read/write/access rights. I don't understand why you should copy the contents of the ./src folder to /usr/bin.
-
purvis: I suggest you to rebuild the squidguard rpm starting from the src.rpm, using the latest tgz available and adjusting the .spec file accordingly
search for "maximum rpm" for more info..
and take a look here (http://www.excaliburtech.net/archives/46)-- maybe you could be interested..
in any case, U S E google, you'll find an answer
-
removed becuase not needed
-
i also read some where on the internet that in version 1.4 where you to place the lines below inside of the squid.conf file, as we do in the /etc/e-smith/templates-custom/etc/squid/squid.conf directory.
redirect_program /usr/bin/squidGuard -c /etc/squid/squidguard.conf
redirect_children 5
that you have to make a change and proceed the words redirect with "url_" like
url_redirect_program /usr/bin/squidGuard -c /etc/squid/squidguard.conf
url_redirect_children 5
but my first success is running without the "url_" addition to the beginning so i do not know if that was correct or not.
This should be in their documentation I think, did you look at that. It might help you (a lot).
-
echo "" > 45squidGuard
echo "redirect_program /usr/bin/squidGuard -c /etc/squid/squidguard.conf" >> 45squidGuard
echo "redirect_children 5" >> 45squidGuard
echo "" >> 45squidGuard
Wouldn't it be easier to just store these fragments somewhere instead of (re)creating them using echo statements? That way you can easily edit them using your favorite text editor. You could easily copy or wget them from somewhere if you need them.
-
removed because not needed
-
Cactus, i did not know how to do make it where a another person could easily do what i wanted to show them.
i figured they could copy that code. yes they make copy and paste errors, but they can copy that.
this is my first coding of showing how i installed it and i was hoping somebody was going to show me another way that is less prone to error.
wget is out of the question for other users, unless you know of a place to store those.
for my purposes, yes, this is not my way of coping files.
i did see some helpful posting on this forum, but a bit old and it never did showed me how to compile code to run on our sme8beta6 machines.
i figured all this would be helpful in the near future when a release comes out.
i am also sure that there are some errors in those lines of code, even as hard as i tried to produce a good first shot on installation.
maybe somebody more skilled in making rpms or however it gets done will take the work i put in and run with it.
-
removed because not needed
-
If I were you I would start with the squidGuard rpm from EPEL - http://download.fedora.redhat.com/pub/epel/5/i386/squidGuard-1.4-8.el5.i386.rpm is the current version - I like http://pkgs.org to search for what software has been packaged already. EPEL and RPMForge both provide quality packages for EL5 (SME 8 beta)
After you install that, whatever changes you have to make to get it working (symlink to start in runlevel 7, changes to the conf files, etc) are what would make up the rest of your howto and what would eventually be what would go into a new smeserver-squidGuard rpm. If changes to any already templated conf files are needed (ie, squid.conf) your howto should instruct people to make changes by using the templates-custom directories - some info here - http://wiki.contribs.org/Template_Tutorial
Good luck
-
Thanks Greg, i will definitely look into that.
-
Ok folks, a review of what i have done and noticed trying to get squidguard 1.4 working.
as i said before, i am not even a qualified person on compiling and/or running squidGuard, just somebody getting trying to get it working.
in no particular order
i reinstalled a successful running of squidGuard off a fresh sme8beta6 about 5 times to make sure the scripts above worked.
compiling did not seem to be an issue, although some how i got two different sized files of squidGuard on my test machines and that was beyond me.
i did all kinds of stuff over the last week and lost track of what i had done but in the end came up with the above scripts.
i believe one file came from a download site off the internet.
the size i got with the down load from squidguard.com with then command
wget 'http://www.squidguard.org/Downloads/squidGuard-1.4.tar.gz'
and after compiling produced a squidGuard size of 278101 that is now my running program.
the only two things i needed to install was the compiler gcc and db4-devel to create and run the squidGuard file.
i think db4-devel is not even needed after a successful compile, because i removed it "yum -y remove db4-devel" in a test situation.
i used as much code as i could find from this forum and the internet and also tried to keep some standards that he sme server is built on.
which means custom templates.
on the squidGuard website, you might see where if you are using squid version 2.6 to use new words that are different in the squid.conf file than previous versions, or at least that is what i am reading into it
see http://squidguard.org/Doc/verify.html at the bottom of the page
redirect_program /usr/bin/squidGuard -c /etc/squid/squidguard.conf
redirect_children 5
would become
url_rewrite_program /usr/bin/squidGuard -c /etc/squid/squidguard.conf
url_rewrite_children 5
this might work but old wording is working now and i have been through enough testing for now.
i had some other problems too.
in an effort to produce fragments of the custom template squidguard.conf in the directory of /etc/e-smith/templates-custom/etc/squd/squidguard.conf.
i had not put in an so called escape character before special characters.
i thought by using the command "expand-template /etc/squid/squidguard.conf" that the program building squidguard.conf in the /etc/squid directory would have just read my lines as they where, line for line. But that was not the case. Whatever program reads the file fragments from the template-custom directory does not treat all characters the same.
With this in mind, it might prove better to create a perl script to read the template-custom files and create the /etc/squid/squidguard.conf. That way one could create a squidguard.conf file from files other than just unix text files.
so from the standard sme way of doing things, watch your special character handling in those fragment files and do not put special characters that you do not have to into custom template files where expand-template is being used.
using of db files,
i do not know what is going on here but all i know is that when i created and made use of the db files, it caused squidGuard not to work. This is only taking into consideration the installing of squidGuard setup that is from the above script
once i deleted the db files, squidGuard worked. When i created them, squidGuard did not work.
so i was following other internet howtos on using the command "squidGuard -C all" to create db files. inside the squidguard log fil "/var/log/squidguard/squidGuard.log", after using the command, all seemed ok, but it stopped squidGuard from becoming active and i also though it even broke squid from running proper or not at all.
well buddy, i can tell you, i am taking baby steps and if i can use squidGuard now without using those db files, that is what i am going to do. i am sure in time with using squidGuard that i will want to use db files, but now know if it is causing my setup not to work.
key words
there may also be a possible problem concerning key words inside the squidguard.conf file such as dest and acl, and other like keywords. i thought i broke the loading of squidGuard by not having some of these key works left justified and it squidGuard is that picky about formatting. the program may be more picky about how many spaces are used and where the curve brackets are placed and other such formatting. like i said above in the custom template building, do not place no more special characters in these files other than alphanumeric if can do so.
the squidguard.conf file in the /etc/squid/ directory
i also found out that signal-event post-upgrade command does not create the /etc/squid/squidguard.conf automatically. i do not know if it ever did.
signal-event post-upgrade did not erase the /etc/squid/squidguard.conf file either.
in order to create the file squidguard.conf in the /etc/squid directory, you either need to copy a squidguard.conf file you have created or place fragments of the file in /etc/e-smith/templates-customer/etc/squid/squidguard.conf directory and use the command "expand-template /etc/squid/squidguard.conf" to create the file from fragments or a single file in the /etc/e-smith/templates-custom/etc/squid/squidguard.conf/ directory.
-
Purvis, please do not remove the content off your posts, even if the instructions are no longer needed. Perhaps it is better to document this in the wiki then in the forums. You can start by draftin on your user page and once in a suitable state move it to the proper namespace.
-
Ok Cactus, thanks, i have submitted a wiki membership
Is there anyway for contribs.org to host files?
after many trials and working on perfecting scripts, i did not want some web surfer to not get the best coding possible or
get the wrong or old stuff, or mislead somebody.
i am now working on trying to get the squidGuard 1.4 program to work with Berkeley db files.
i do not think i have seen a how to do that here on this forum and other web pages have links to sites that have long disappeared.
it is also very nice to use the most current release coding of a program when available.
this link below looks promising and i have done most the work, but no final success.
http://felipeferreira.net/?p=240
-
ok, now i have the instructions for installing the Berkeley DB data base lookup.
Maybe i had some errors in the top too, i do not know.
I believe i should have done the command "chown -R squid /var/lib/squidguard/db" especially if a command of "squidGuard -C all"
and with that might of done some good but i did not have the Berkeley DB installed.
with the below code, i found that i could not compile to squidGuard and produce the executable squidGuard program file if i installed Berkelely DB first.
so please remember to compile squidGuard first. and what seemed really weird is that as long as i compiled and created squidGuard first then i could produce and compiled squidGuard after i installed Berkeley Db also.
so just compile squidGuard first before Berkeley DB
also during my testing, one version of the squidGuard program could not find the file libdb-4.6.so in the BerkeleyDB lib folder so i copied it into the folder /usr/lib and everything went back to working, but that was during testing and the code below seems to work great as it is, but i wanted to mention it if you see that statement in an error from squidGuard.
so here is the code below as i have it
-
log into the sme server with putty
run the below
this first step of instructions just creates some directories, in the case some goes wrong with your coping and pasting in putty
cd /
mkdir -p /temp
mkdir -p /var/lib/squidguard/db
mkdir -p /var/log/squidguard
mkdir -p /etc/e-smith/templates-custom/etc/squid/squid.conf
mkdir -p /var/lib/squidguard/db/dame
mkdir -p /etc/e-smith/templates-custom/etc/squid/squidguard.conf
cd /
cd /
mkdir -p /temp
cd /temp
wget http://www.xoroz.com/files/db-4.6.21.tar.gz
sleep 15
wget 'http://www.squidguard.org/Downloads/squidGuard-1.4.tar.gz'
sleep 15
tar -zxvf squidGuard-1.4.tar.gz
sleep 15
tar -zxvf db-4.6.21.tar.gz
cd /
sleep 15
cd /
yum -y install bison
sleep 15
yum -y install flex
sleep 15
yum -y install db4-devel
sleep 15
yum -y install gcc
sleep 15
signal-event post-upgrade
sleep 15
signal-event reboot
cd /
give your sme server some time to stablize after booting back up
i run the command "yum clean all" and look for something like below:
Loaded plugins: fastestmirror, protect-packages, smeserver
Cleaning up Everything
after a reboot then log back into the sme server with putty
run the below
this part will compile and create squidGuard and place the executable file in the /usr/bin directory
it is fairly fast
just a handy note:
on my computer inside putty, during this compile process i was able to see the file squidGuard in GREEN COLOR.
so just look for that (squidGuard file in green color) and that would confirm the squidGuard program was created during the compile.
cd /
cd /temp/squidGuard-1.4
./configure -prefix=/usr/bin --with-squiduser=squid --with-db-lib=/usr/local/BerkeleyDB/lib --with-db-inc=/usr/local/BerkeleyDB/include --with-db=/usr/local/BerkeleyDB --with-sg-config=/etc/squid/squidguard.conf --with-sg-logdir=/var/log/squidguard --with-sg-dbhome=/var/lib/squidguard/db
sleep 5
make
ls /temp/squidGuard-1.4/src
sleep 20
cd /
cp /temp/squidGuard-1.4/src/squidGuard /usr/bin
chmod 755 /usr/bin/squidGuard
mkdir -p /var/lib/squidguard/db
mkdir -p /var/log/squidguard
rm -rf /var/log/squidguard/squidGuard.log
cp /etc/squid/squidguard.conf /etc/squid/squidguard.conf.new
rm -rf /etc/squid/squidguard.conf
chown -R squid /var/lib/squidguard/db
cd /
this part will compile BerkeleyDb and install the files in the needed place ton the server in the /usr/local/BerkeleyDB directory
this part will produce a lot of output of characters to screen and take a longer than expected but not tooo long.
cd /
cd /temp/db-4.6.21/build_unix
../dist/configure -prefix=/usr/local/BerkeleyDB
sleep 10
make
sleep 10
make install
sleep 5
dir /usr/local/BerkeleyDB
cd /
this will create some other files needed and you may have a few bell sounds occur and i do not know why but
it appears to run as it should, so do not be alarmed by any bells produced in this routine.
cd /
mkdir -p /etc/e-smith/templates-custom/etc/squid/squid.conf
cd /etc/e-smith/templates-custom/etc/squid/squid.conf
echo "" > 45squidGuard
echo "redirect_program /usr/bin/squidGuard -c /etc/squid/squidguard.conf" >> 45squidGuard
echo "redirect_children 5" >> 45squidGuard
echo "" >> 45squidGuard
sleep 3
cd /
expand-template /etc/squid/squid.conf
sleep 5
mkdir -p /var/lib/squidguard/db/dame
echo "# write domains you'd like to prohibit access to" > /var/lib/squidguard/db/dame/domains
echo "yahoo.co.jp" >> /var/lib/squidguard/db/dame/domains
echo "example.com" >> /var/lib/squidguard/db/dame/domains
mkdir -p /var/lib/squidguard/db/dame
echo "# write URLs you'd like to prohibit access to" > /var/lib/squidguard/db/dame/urls
echo "www.yahoo.co.jp/deny/" >> /var/lib/squidguard/db/dame/urls
echo "www.example.com" >> /var/lib/squidguard/db/dame/urls
chown -R squid /var/lib/squidguard/db
mkdir -p /etc/e-smith/templates-custom/etc/squid/squidguard.conf
cd /etc/e-smith/templates-custom/etc/squid/squidguard.conf
echo "#" > 10abegin
echo "# CONFIG FILE FOR SQUIDGUARD" >> 10abegin
echo "#" >> 10abegin
echo "" >> 10abegin
echo "dbhome /var/lib/squidguard/db" > 10adbhome
echo "logdir /var/log/squidguard" > 10alogdir
echo "" > 20destdame-a
echo "# define a category dame" >> 20destdame-a
echo "dest dame \{" >> 20destdame-a
echo " # define prohibit domain file for dame" >> 20destdame-a
echo " domainlist dame/domains" >> 20destdame-a
echo " # define prohibit URL file for dame" >> 20destdame-a
echo " urllist dame/urls" >> 20destdame-a
echo "\}" >> 20destdame-a
echo "" >> 20destdame-a
echo "acl \{" > 20destdame-b
echo " default \{" >> 20destdame-b
echo " # allow except dame" >> 20destdame-b
echo ' pass !dame all' >> 20destdame-b
echo " # redirect URL" >> 20destdame-b
echo " redirect http://localhost" >> 20destdame-b
echo "\}" >> 20destdame-b
echo "\}" >> 20destdame-b
echo "" >> 20destdame-b
rm -rf /etc/squid/squidguard.conf
expand-template /etc/squid/squidguard.conf
sleep 10
rm -rf /var/log/squidguard/squidGuard.log
this part restarts you squid and makes the squidGuard program active without having to reboot and
it is necessary because the squidGuard program creates some database files for faster searching of
websites located in your rules listing.
the chown line is very important after building those database files to give squidGuard permissions to access those the database files created.
without the chown command, your squidGuard is likely to not work and it might even cause squid not to work.
clear squid cache
cd /
svc -d /service/squid
sleep 10
echo "" > /var/spool/squid/swap.state
squidGuard -C all
chown -R squid /var/lib/squidguard/db
sleep 10
/sbin/service squid restart
cd /
done with processing, check on program
now you should be able to do "squidGuard -d" to see if your program is
with no negative results working properly
squidGuard -d
now in a browser, with its proxy configuration pointing to this server, you
can go to the websites of yahoo.co.jp and/or example.com and it should point back to
your server where this is installed.
-
i did use this reference a lot while working on the code, it is a good reference.
http://felipeferreira.net/?p=240
in this reference, you will notice the squidGuard program was compiled first before BerkeleyDB.
also, the notations of somebody mentioning to change the redirect_program and redirect_children wording inside the
the squid.conf to url_rewrite_program and url_rewrite_children.
see squid.org for details on the wording
also notice that i do not let squidGuard install the program with (make install) into my system directories.
but i do let the BerkeleyDB install to my system directories.
also pay attention when installing BerkeleyDB and install it only once, if you try to install it a second time
you will have run some programs to clear out some stuff that was created during the first compile.
the info on that is listed in the referenced linked website in this post above
some other notes:
as far as i can tell, if you have db files created in the /var/lib/squidguard/db folders and you do not have BerkeleyDB support,
then your squidguard program is likely not to work. that has been what i have seen and i again am only a beginer with using squidGuard.
How i got my squidguard program to running was by removing the db files or possibly just renaming them, which i have not tried.
More that likely though you are going to want to start making use of the blacklists db files that can be downloaded from the internet as so will i.
i did see on this form where some people where running a command "config show squidGuard" to get some feed back.
i think running the command "squidGuard -d" may possibly be your best form to check to see if the squidGuard program is running.
a very likely better form of a test, could actually be a routine to run from a cron that creates a report from trying to access either one or mulitple
webpages and the webpage is redirected to a file and that file is read in an automated way, then analyzed on the results that some get your
in a way that the squidguard routine seem broken.
Many people on this forum also suggest that you uninstall the gcc compiler after your done using it.
that can be done with a "yum -y remove gcc" then "signal-event post-upgrade;signal-event reboot" giving a immediate reboot of the sme server.
you may want to remove the directory /temp or rename it, but for me, i am going to leave those files on my server for a while
-
You have two post regarding SquidGuard and SME 8. This seems to be the latest one. Is this the only development with SquidGuard and SME 8?
What about the EPEL5 rpm? Is anyone using that, or is purvis the only one using SquidGuard?