+ Post New Thread
Results 1 to 14 of 14
*nix Thread, squid transparent proxy cache questions in Technical; im attempting to use squid in transparent proxy mode simply as a cache for a slow internet link. Dont need ...
  1. #1

    RabbieBurns's Avatar
    Join Date
    Apr 2008
    Location
    Sydney
    Posts
    5,531
    Thank Post
    1,341
    Thanked 470 Times in 307 Posts
    Blog Entries
    6
    Rep Power
    200

    squid transparent proxy cache questions

    im attempting to use squid in transparent proxy mode simply as a cache for a slow internet link. Dont need any filterring / logging / authentication etc.
    • Do i actually need 2 network interfaces?
    • Do i need 2 network cards or can I create a virtual sub interface?
    • Can squid cache youtube? Ive read mixed reports.
    • Is 20GB enough for a cache for 60 users? Is 512MB RAM enough for the box running ubuntu server?


    Cheers

  2. #2

    dhicks's Avatar
    Join Date
    Aug 2005
    Location
    Knightsbridge
    Posts
    5,683
    Thank Post
    1,268
    Thanked 789 Times in 686 Posts
    Rep Power
    237
    Quote Originally Posted by RabbieBurns View Post
    Do i actually need 2 network interfaces?
    If you're not using the Squid proxy for filtering and you're not worried about people being able to bypass the server then no, you can just set the server as a proxy in your browser settings (or set that via group policy for the whole domain).

    Do i need 2 network cards or can I create a virtual sub interface?
    I forget, offhand, exactly how to best set u Squid with one network card, but I know there's an easy-to-follow how-to. However, with network cards casting around £10, it'd proably be simple enough just to add a second network card and be done with it.

    Can squid cache youtube? Ive read mixed reports.
    I don't see why not, although I might be missing something - YouTube uses FLV files to display video, which are proper files capable of progreessive download, not streamed. Of course they might now have some kind of streaming service, you'd have to check.

    Is 20GB enough for a cache for 60 users? Is 512MB RAM enough for the box running ubuntu server?
    Not sure about the disk space - it does seem rather small, can't you just give your machine a larger harddrive? I find Debian, and therefore probably Ubuntu Server, runs okay in 512MB of RAM, although it struggles doing anything much with less than that.

  3. #3


    tom_newton's Avatar
    Join Date
    Sep 2006
    Location
    Leeds
    Posts
    4,485
    Thank Post
    867
    Thanked 854 Times in 675 Posts
    Rep Power
    197
    No, and No. 1 interface fine.
    Yes you can cache youtube, but not out of the box. This is against youtubes T&C.
    20GB is far too much. You will spend too much CPU time managing it, and it will be full of stale data. 2-5Gb more sensible.

  4. 2 Thanks to tom_newton:

    dhicks (19th June 2012), RabbieBurns (20th June 2012)

  5. #4


    Join Date
    Jan 2006
    Posts
    8,202
    Thank Post
    442
    Thanked 1,032 Times in 812 Posts
    Rep Power
    339
    Quote Originally Posted by RabbieBurns View Post
    i[*]Do i actually need 2 network interfaces?[*]Do i need 2 network cards or can I create a virtual sub interface?[*]Can squid cache youtube? Ive read mixed reports.[*]Is 20GB enough for a cache for 60 users? Is 512MB RAM enough for the box running ubuntu server?[/LIST]

    Cheers
    20GB - easily enough!

    You don't need two interfaces.
    We put the Squid cache in a DMZ.
    The DMZ is accessible through the internal VLAN's
    The Firewall has an exception for quid to access the internet, and for the internal networks to access the cache.

  6. 2 Thanks to CyberNerd:

    RabbieBurns (20th June 2012), tom_newton (19th June 2012)

  7. #5

    Geoff's Avatar
    Join Date
    Jun 2005
    Location
    Fylde, Lancs, UK.
    Posts
    11,813
    Thank Post
    110
    Thanked 586 Times in 507 Posts
    Blog Entries
    1
    Rep Power
    225
    VLANs would be another option if you wanted to use one network card but needed some sort of separation of traffic flows.

    Youtube caching would work fine (although I'd like to hear what the issue is with the T&Cs and caching from @tom_newton).

    20gb is fine, but I've only used such big caches when I needed to cache big files for a while (eg windows update with no local WSUS and slow broadband).

  8. Thanks to Geoff from:

    RabbieBurns (20th June 2012)

  9. #6
    morganw's Avatar
    Join Date
    Apr 2009
    Location
    Cambridge
    Posts
    816
    Thank Post
    46
    Thanked 132 Times in 126 Posts
    Rep Power
    40
    Remember that you cannot transparently proxy HTTPS without doing SSL interception with your own certificate. Everyone could get quite confused if "the internet" breaks everytime they try to login to a site.

  10. #7


    tom_newton's Avatar
    Join Date
    Sep 2006
    Location
    Leeds
    Posts
    4,485
    Thank Post
    867
    Thanked 854 Times in 675 Posts
    Rep Power
    197
    Quote Originally Posted by morganw View Post
    Remember that you cannot transparently proxy HTTPS without doing SSL interception with your own certificate. Everyone could get quite confused if "the internet" breaks everytime they try to login to a site.
    Actually you can... if you're willing to sacrifice a bit of accuracy in places...

  11. #8

    RabbieBurns's Avatar
    Join Date
    Apr 2008
    Location
    Sydney
    Posts
    5,531
    Thank Post
    1,341
    Thanked 470 Times in 307 Posts
    Blog Entries
    6
    Rep Power
    200
    this is just for a remote site with a very slow internet connection, with an unmanaged 3com switch, a couple of APs and 30 laptops.

    I want transparent - not to set any browser settings for proxy. I just want to drop this box in between the switch and the internet to be a simple webcache, and the laptops use this as thier Default GW.

    Not fussed if 443 isnt cached. Its just really to save bandwidth and speed up browsing.

    20GB is the size of the entire HDD

    Thanks for letting me know youtube will work. Had read that due to the dynamic CDN and random strings in the URLs it didnt work.

  12. #9

    RabbieBurns's Avatar
    Join Date
    Apr 2008
    Location
    Sydney
    Posts
    5,531
    Thank Post
    1,341
    Thanked 470 Times in 307 Posts
    Blog Entries
    6
    Rep Power
    200
    Ive just thought of a flaw in my plan. The reason Ive said I wasnt concerned about filtering/logging/authentication is because we have a UTM appliance box that does all this. But this does it based on IP address and usernames, however if this box sees all the web traffic only coming from 1 IP (the squid box) its going to break authentication etc.

    Is there any way with squid to pass the source IP from the client through the proxy to the UTM appliance?

  13. #10


    tom_newton's Avatar
    Join Date
    Sep 2006
    Location
    Leeds
    Posts
    4,485
    Thank Post
    867
    Thanked 854 Times in 675 Posts
    Rep Power
    197
    Yes - if the UTM supports it - you can pass the x-forwarded-for IP out of squid - this may or may not fix your authentication, depending on the type. NTLM & Kerberos can sometimes be passed through to an upstream proxy, and captive-portal based auth can work if the xff IP is honoured.

    Does your UTM not do caching anyway?

  14. Thanks to tom_newton from:

    RabbieBurns (20th June 2012)

  15. #11

    RabbieBurns's Avatar
    Join Date
    Apr 2008
    Location
    Sydney
    Posts
    5,531
    Thank Post
    1,341
    Thanked 470 Times in 307 Posts
    Blog Entries
    6
    Rep Power
    200
    the UTM is at the other end of an IPSec VPN over a slow ADSL. I want to put a local cache at the slow end. It will be NTLM auth as I dont think the FSAE collector works over the VPN.

  16. #12


    tom_newton's Avatar
    Join Date
    Sep 2006
    Location
    Leeds
    Posts
    4,485
    Thank Post
    867
    Thanked 854 Times in 675 Posts
    Rep Power
    197
    As NTLM is connection based you might get away with it - I think there is a spot of tweaking and prodding you may have to do with squid, but can't remember the details...

  17. Thanks to tom_newton from:

    RabbieBurns (20th June 2012)

  18. #13

    RabbieBurns's Avatar
    Join Date
    Apr 2008
    Location
    Sydney
    Posts
    5,531
    Thank Post
    1,341
    Thanked 470 Times in 307 Posts
    Blog Entries
    6
    Rep Power
    200
    I have got squid up and running in transparent mode on one interface working great. However the problem I had envisaged is as expected.

    I have set the X-Forwarded-For directive to Truncate, but the UTM (its a Fortinet device) doesnt seem to recognise it. I get the NTLM popup for authentication the first time a client tries to access the internet, but this just associates that user with the IP of the squid box and every subsequent client is logged against this initial user.

    Am I at a dead end here or is there anything I might be able to do to have a webcache in place without using it as a proxy ?

  19. #14

    Join Date
    Jan 2010
    Posts
    64
    Thank Post
    7
    Thanked 5 Times in 5 Posts
    Rep Power
    10
    how did you get on with youtube caching? as want to look at doing it here

SHARE:
+ Post New Thread

Similar Threads

  1. Transparent Proxy with LEA supplied squid server
    By CHR1S in forum Internet Related/Filtering/Firewall
    Replies: 15
    Last Post: 15th May 2012, 10:45 AM
  2. Squid transparent proxying
    By MK-2 in forum *nix
    Replies: 46
    Last Post: 4th June 2008, 11:26 AM
  3. Squid Transparent
    By Jackd in forum *nix
    Replies: 5
    Last Post: 10th October 2007, 02:12 PM
  4. Squid Transparent Proxy.
    By Jackd in forum Network and Classroom Management
    Replies: 2
    Last Post: 25th July 2007, 06:54 PM

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •