Topic: recompile squid

[Tom Haynes]

Greetings...

I am trying to recompile squid to improve performance.

There is a howto at http://www.linux-kita.com/data/Optimizing-Squid-for-Linux-2.4-HOWTO.html

I installed dev-tools and the source for the e-smith squid on a 5.6 test machine.

I followed the instructions to edit /usr/include/bits/types.h, and I went to the directory to run "ulimit -HSn 8192." When I run ./configure, it gives an error of

"checking Default FD_SETSIZE value... 256
checking Maximum number of filedescriptors we can open... 256"

This was not the intent at all.

Does anyone have an idea of what I am doing wrong?

Regards...   Tom

[%sig%]

[Charlie Brady]

Tom Haynes wrote:

> I am trying to recompile squid to improve performance.

Have you first determined that squid is limited in its performance? Do you really have c. 50 requests per second?

> Does anyone have an idea of what I am doing wrong?

Sorry, no.

Charlie

[Tom Haynes]

We are a laptop school.

We have almost 1000 users online at one time at peak. The machine is running e-smith 5.6 with the tech-geeks squidGuard. It is a dual processor 900 PIII with scsi and 2 gig of ram.

When traffic is low, the proxy is pretty snappy. When traffic is high during study hours or right after school, it grinds to a halt. I can use another proxy on the internal network and get good performance when everything going through the main proxy is dragging.

We have a cron job that clears the cache and restarts squid every half hour on the main proxy to keep it functional. Really this is not often enough at peak and overkill the rest of the time.

Checking the squid FAQ and the squid list archive, we need to increase the number of filedescriptors.

I figured someone might have done this before or have some pointers.

[%sig%]

[Graeme Fleming]

It would seem that the main prob you have is disk performance.  Pickup a s/h caching SCSI HA (Adaptec 2100S for example) and this should pep things up nicely.

Also look at James Prices' contrib for Squid - more usable than the Tech Geeks.

Check contrib.org for a FAQ on setting up multiple proxies when using multiple paths to the internet, it might give some clues.

HTH

[marco]

if you need a squid rpm that will allow more file descriptors, you can download it here:

http://swelltech.com/support/updates/squid/7.x/RPMS/

there are a few to choose from.  After installing the rpm, you must edit the /etc/init.d/squid file and add the following items in right after the start line in the file:

echo 8192 > /proc/sys/fs/file-max
ulimit -HSn 8192

You can go as high as 65536 for both numbers, but only 32768 file descriptors will be given.  Make sure you have enough processor power and at least 768MB of ram.  I would also suggest setting the cache directory to a uw160 scsi drive so that you have fast disk response.

Hope this helps

[Tom Haynes]

Thanks...

I'll give this a try tomorrow.

I appreciate the help.

[Arne]

Just one "stupid" idea .. (I have no experience running networks at that size) What about setting up two (or more) sme servers in paralell ? You wuld actually need minimum 3 PC, one in front to chear the trafic, this could be anything with a Linux 2.4 kernel, a RedHat minimum installation as an excample. Behind that firts nat router you could set up 2 or more sme servers. By selectiong different gateway ip's on the clients you could chear the trafic between the 2 "inner" sme gateway proxyes in such a way that 500 would pass trought no1 and 500 trough no 2. I believe such a "battery" of sme proxy server should be able to do the job (?) at least from a theoretical point of wiew.


--(Gateway 1) ---------------------(sme proxy 21)---------------------(LAN)
                                            I
                                            ------(smeproxy 22)-----------------------(LAN)


It should be possible to connect it all to the same LAN segment, if requiered.

[Arne]

--(Gateway 1) ---------------------(sme proxy 21)---------------------(LAN)
xxxxxxxxxxxxxxxxxxxxxxxxxxx I
xxxxxxxxxxxxxxxxxxxxxxxxxxx ------(smeproxy 22)-----------------------(LAN)

[Marco Garza]

Tom,
What version of SME are you using?   What SSL rpm did you use?

[Tom Haynes]

I installed "openssl096-0.9.6-6.i386.rpm" from rpmfind on my test 5.6 box at home. I then installed the swelltech "squid-2.4.STABLE7-1rh.i386.rpm"

I modified the init script to include

echo 8192 > /proc/sys/fs/file-max
ulimit -HSn 8192

I restarted squid and checked "ulimit -a"

I am at 1024 filedescriptors. If I run the same two lines from a command prompt, "ulimit -a" gives me 8192...

This test box had seen a lot of testing, and I decided to start fresh with a new 6.0 install.

I went through the same procedure on the 6.0 box with the same rpms. I got the same results.

[%sig%]

[Tom Haynes]

For those who are interested in this thread, I got some help from Marco on this off list, and have increased the number of file desciptors for squid. I had this installed on a developmental (retired, low spec) server, and we used it as the main proxy yesterday afternoon and night. All worked very well.

We updated the main proxy to the new config, made the switch to the updated machine, and we seem to have solved the problem.

I will probably write this up as a howto, but it may have to wait until Christmas Break.