Squid transparent proxying

[MK-2]

I know there are already threads on the subject, but they seem from a more "I've set it up now need to tweak it" angle. Mine is from a "I'm stupid and need help" angle

 

Got a box here I'm going to install Ubuntu on, and set it up with Squid/DansG for proxy and filtering.

I was told that for transparent proxy you need two network cards, which we have.

How exactly is it setup, because thanks to our EMBC filtering I can't find any guides as its banned due to proxy anonymizer!!!

Is it set up like

eth0=192.168.1.2

eth1=192.168.1.3

and you get all traffic redirected to eth0 from eth1?

I'm just after a comprehensive guide on setting up squid transparently to work properly.

Somebody else told me it had to be compiled with a transparent switch or something, but I'm not sure.

Any help is much appreciated, ta

[Geoff]

You do not need two network cards. It will work with one. However for performance reasons you are better off with an 'in' and an 'out' card.

 

Your Linux box needs to be the default gateway for your client machines *OR* configured as a bridge and 'inline' between your client machines and your current gateway.

[gaz350]

someone correct me if im wrong.

The transparent part is done using iptables at the gateway to redirect port 80 traffic to the proxy ip:port.

 

so if your not doing it at the gateway it wont work.

[MK-2]

What we have, and what we are looking at doing is:

 

One server

BT Broadband + router

3 Wireless APs

One switch

100 PDAs

This will all be a separate network to our curriculum one.

 

What we want are for the PDAs to connect to the internet via the server, and be filtered.

So what's best, the bt router plugs into the switch, as do both the NICs on the server. Do I then need to update iptables or something to tell it where to redirect traffic for it to be transparent?

 

This is why I need a sort of step by step guide on a webpage somewhere. I know how to use linux casually, but not how to set up squid and stuff

[ICT_GUY]

This might help.

 

Setting up Squid as your caching HTTP/FTP proxy

 

By Trevor Warren

Posted: ( 2000-10-23 07:35:41 EST by )

Squid is a proxy caching server for HTTP/FTP requests. It caches data off

the net on your local network. So the next time the same data is being

accessed, whether it is html or a gif, it gets served up from the local

server rather than over the Internet -- saving you significant bandwidth.

 

Lets use the most commonly available proxy server for Linux and the most

stable one around, Squid. Installing and configuring it is a breeze as

you'll soon find out. To make things simpler I would suggest that you get

the Squid RPM from any of the download on the net for your distro.The

latest Stable release of Squid is squid-2.3.STABLE1-5.i386.rpm. If you are

not able to find it on your distro's CD then i would suggest you try out

Rpmfind mirror. After having downloaded the RPM install it with the

following command.

 

Assuming you have downloaded the squid-2.3.STABLE1-5.i386.rpm release

the installation command is as follows.

 

bash# rpm -ivh squid-2.3.STABLE1-5.i386.rpm

 

And please do note that "bash#" stands for the shell prompt and you do not

 

need to replicate it in your command.

 

Having installed Squid sucessfully, now open the file /etc/squid.conf

using your favourite text editor. Some distributions put this file in

/etc/squid/. This is where it gets interesting and confusing too so read

carefully.

 

Scroll down till you come to the line

 

#http_port 3128

 

This option sets your HTTP proxy port to 3128 which is the default port

that squid runs on. You can uncomment this line and set it to whatever

port you want. It is advisable to avoid port 80 since, if you are running

a Web Server on the Linux machine Apache would be listening on that port.

 

Scroll down till you come to the line

 

#cache_mem 8 MB

 

This option sets a limit on the amount of memory that squid may make use

of to store it's transient and cached objects temporarily in memory. This

limit that you may impose on squid is the soft limit and at any given

point of time Squid may double or triple the size of occupied memory all

depending on the size and the requirement of in-transient cached objects.

Uncomment this line and change the size of your Memory cache from 8 MB to

what ever size you want it too be. Keep in mind the amount of RAM that you

have on your machine when you allocate memory to SQUID. But for your

knowledge this occupancy of the specified Memory Limit is dynamic.

 

Scroll down until you come to the following lines

 

# LOGFILE PATHNAMES & CACHE DIRECTORIES

#--------------------------------------------------------------------------

 

The following options are relative to setting up and tuning your web

cache. So lets get gunning pals. Here the first and most important one.

 

#cache_dir /var/squid/cache 100 16 256

 

Isn't this getting a little confusing, one parameter and three values to

it! C'mon let's demystify the whole thing. The values given here are the

values the Squid will use by default. So if the 'cache_dir' option isn't

implicitly mentioned then Squid resolves to maintaining the cache in

/var/squid/cache. Uncomment this option 'cache_dir' if you want to

customize the parameters.

 

The first parameter '/var/squid/cache' is the path to the cache files. You

may change this to suit whatever you want too, but remember whatever path

you may mention out here make sure that those directories exist because

Squid will never create the directories on it's own. Also a point to be

noted is that the directories should be writable by the Squid process. If

you are a novice and all of this is sounding too geekish then I suggest

you stick to the default values.

 

The next value '100' is the amount of space in MegaBytes(MB) that Squid

can utilize to store the cache contents. Modify this to whatever you think

is appropriate to suit your needs.

 

The next value referred to as 'Level-1' is the number of sub-directories

that Squid can create under the current Cache directory. I suggest that

for starters leave this as it is.

 

The next option is referred to as 'Level-2' is the number of Second Level

directories that Squid can create under each 'Level-1' directory. The

default is fine for the moment.

 

Scroll down till you come to the line:

 

# ACCESS CONTROLS

# -----------------------------------------------------------------------------

 

The following lines define Access Control Lists for your Network. Squid

allows you to define various kinds of ACL's out here. So make it a point

to read this whole section of Access Controls carefully.

 

In this "ACCESS CONTROLS" section scroll down till you come to the

following lines.

 

#Default configuration:

http_access allow manager localhost

http_access deny manager

http_access deny !Safe_ports

http_access deny CONNECT !SSL_ports

#

# INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR

# CLIENTS

#

http_access deny all

 

What you need to do out here, is to setup your own ACL's (Access Control

Lists), else just comment out the last line as shown above and put the

following line in.

 

http_access allow all

 

So now your rule section should look like this.

 

# INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR

# CLIENTS

 

#http_access deny all

http_access allow all

 

Three cheers and your proxy has been setup. Now you only need to make sure

that Squid starts every time your Linux box boots.

 

If your using RedHat then you can start in the following manner. Login as

Admin or "su" to root & use the "setup" command. Now enter the System

Services sub menu and enable Squid.

 

If your using SuSE then start YaST and go to "System Administration", then

go to "Change config File", then scroll down till you come to "START

SQUID" & just enable it from "NO" to "YES".

 

The next time you reboot your machine your Proxy will start automatically.

 

Before you start using squid you need to create the swap directories. Do

this by

 

/usr/sbin/squid -z

 

This just has to be done the first time.

 

To start Squid right now use the following command

 

bash#/etc/rc.d/init.t/squid start

 

There, you have setup, configured and started your proxy. Just make sure

your client's web browsers have the http proxy port setup as the same port

in your /etc/squid.conf file.

 

Squid Home Page

 

Other articles by Trevor Warren

 

Any good?

[MK-2]

ICT_GUY thats a good help, cheers, but what about the transparent bit?

Is that like gaz says, in iptables?

If so, how would I set that out?

 

Am I best doing like i said with eth0/1 and setting them as 192.168.1.2 and .1.3?

[Geoff]

iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 80 -j REDIRECT --to-port 3128

[OutToLunch]

Linux: Setup a transparent proxy with Squid in three easy steps

 

Nice guide here - if you can't access it, give me a shout and I can lob it into a PDF and email it to you.

 

Just a note on the physical setup you describe, you should have something like

 

( Internet ) <-> Router <-> Eth0 on Proxy <-> Eth1 on Proxy <-> Switch <-> Wireless

 

not both cards plus the router plugged into the switch or you will make your proxy easy to circumvent...

[MK-2]

My final question (honest)!

I now have the choice between Ubuntu server or desktop.

I know ideally server would be best, but seeing as the only job this box will be doing is proxy/filtering, is there any major downside to installing the desktop version instead?

Basically will everything still work 100% if I install the desktop version? Other people may need to access the server to update whitelists on filters etc so a GUI would be handier for them so I'd prefer desktop if it will all still work.

Thanks to all of you, we've got the BT router and APs up, it's now just a case of getting the server up and running to filter it all

[Geoff]

Server. Your fellow techies are going to have to use the Linux command line sooner or later, so it's a good excuse to start learning. However if you are feeling charitable then install Samba on the machine and expose the config directory as a share so they can edit the lists from their Windows PCs. They will have to use a text editor that understand *nix format text files though.

[MK-2]

Geoff that's the thing. I don't want to adopt the 'f**k you, learn it' attitude because all that will happen is that nobody does as I can, meaning more work for me.

[el8linuxel8]

just tell them to read man pages and RFC's during their lunch

[Geoff]

That wouldn't fly round here. We need to be able to cover each others jobs in case one of us is sick or gets run over by a school bus.

[OutToLunch]

Install Webmin on the proxy - would be a good compromise...

 

Webmin

 

Will give a web based interface to edit the conf files if necessary, accessible from any machine with a web browser.

[Geoff]

Webmin doesn't work properly on Ubuntu, the filesystem layout is non-redhat. One should use eBox instead.

 

https://help.ubuntu.com/community/eBox

[MK-2]

i'll go with the server and ebox for now. it was me who suggested this whole solution so i guess its only fair i do the admin for it for now.

but thanks for ebox/webmin idea, i guess i could always do an smb share to the whitelist folder and let people update it themselves

cheers guys

[OutToLunch]

Webmin doesn't work properly on Ubuntu, the filesystem layout is non-redhat.

 

Oops Hands up who lost forgot to pay attention to the OPs finer details and just repeated what they used on their CentOS box!

[ICT_GUY]

Ok a bit of a break in replying.

 

After a lot of thinking, playing around with stuff and talking to the local top geeks in the area. I have decided to go with ISA server on Server 2008.

 

Total licence cost is around the £90 mark (love education licences, got to confirm this btw) and it just works. Not even that difficult to set up.

 

Its certainly easier to get working than squid was and so far has not caused any issues.

[mitcheln]

Geoff that's the thing. I don't want to adopt the 'f**k you, learn it' attitude because all that will happen is that nobody does as I can, meaning more work for me.

 

Hi,

Editing dansguardian whitelists is not terribly difficult though. Your colleagues need to know which are the appropriate files, how to navigate to them and how to edit them correctly. I use mc sometimes and nano sometimes via ssh. mc is great for this kind of thing, because it's visual. I'm sure your colleagues will get the hang of it quickly. How will they restart the dansguardian service to reload the new lists? I don't know what gui service controls there are for Ubuntu, but typing /etc/init.d/dansguardian restart on the cli is pretty quick imo.

Anyway, hope you have great success with your project.

[Geoff]

/etc/init.d/dansguardian restart

 

Try this instead if your just updating the white/black/grey lists.

 

dansguardian -g

 

This way, current connections don't get dropped on the floor.

[mitcheln]

Try this instead if your just updating the white/black/grey lists.

 

dansguardian -g

 

This way, current connections don't get dropped on the floor.

 

I see my plan to avoid reading the manual worked. Seriously though, thanks, I'll give that a whirl next time I have to.

[MK-2]

me again

 

ok i got the server up and running today, ebox/dansg/squid installed

eth0 connected to bt router, with ip of 192.168.1.2

eth1 connected to switch with ip of 192.168.1.3

 

firstly, with both IPs set, i cant browse the internet and internal hosts, im guessing it uses one ip for both and that wont work. is there any way i can specify eth0 is only to be used for web requests and eth1 is to be used for internet requests?

 

secondly if i forward all traffic from eth1 on port 80 to eth0 on 3128 so it goes via squid, how do i implement dansg? cos it needs an ip/port to listen on, and if i set it to eth1:80 to listen on, wouldn't that then mean that nothing gets redirected, or will it filter then redirect?

 

haven't done any conf file edits yet, so if anybody would be willing to help me out id be very grateful. basically we just want any pda on the 192.168.1.x ip range (set via dhcp on ubuntu) to have access to the internet with dansg and squid in between in transparent mode.

sorry if im going over things more than once, im simple

[mitcheln]

me again

 

ok i got the server up and running today, ebox/dansg/squid installed

eth0 connected to bt router, with ip of 192.168.1.2

eth1 connected to switch with ip of 192.168.1.3

P

 

Hi there,

Quick reply:

I hope I've attached a diagram to this post. Does this match your setup?

If it doesn't, are you able to create a diagram that does?

I can't speak for others, obviously, but it helps me to see things.

Regards, Nic

[MK-2]

mitcheln:

 

Mine goes:

Internet > BT Router > eth0 (192.168.1.2) > Ubuntu server > eth1 (192.168.1.3) > switch > wireless AP > PDAs

 

so all internal traffic (PDAs, wireless APs) would be hitting eth1, and all internet traffic would hit eth0.

But when both are plugged in, and I try to get on either an AP or the net, it hangs, unplug eth0 and i can see the router, unplug eth1 and i can see the internet. Obviously I'll need both to work together.

 

If anybody is available to hit me up on MSN and give me a hand to go through things I'd love it (Keegan style!). Mainly just with getting both cards working in unison, and getting squid/dansg working together with squid being transparent.

[ConTheITGuy]

I am very novice at this myself, but I was under the impression that both cards couldn't be on the same subnet. Perhaps someone can confirm (or otherwise)

 

 

Andy