+ Post New Thread
Page 1 of 2 12 LastLast
Results 1 to 15 of 17
*nix Thread, Squid with CARP load balancing in Technical; Is anyone using Squid with CARP peers? In a nutshell, I have a group of Squid proxy/cache servers working fine ...
  1. #1

    jinnantonnixx's Avatar
    Join Date
    Mar 2011
    Location
    In the Calamatorium.
    Posts
    1,997
    Thank Post
    115
    Thanked 508 Times in 343 Posts
    Blog Entries
    2
    Rep Power
    286

    Squid with CARP load balancing

    Is anyone using Squid with CARP peers?

    In a nutshell, I have a group of Squid proxy/cache servers working fine (on Centos 5), but I want to move away from my primitive (awful) hashing algorithm in my PAC/WPAD files to a proper load balanced system.

    I've read that CARP can provide a solution, but can't get it to work.

    I've gone through the obvious documentation I could find, but I'm frustrated at the lack of explanation of key points (why would you use sibling instead of parents in your CARP array, for instance).

    I'm adding cache_peer lines to my squid.conf, but my peer servers aren't receiving any directed traffic. i.e. I've done something wrong.

    Any pointers?
    Last edited by jinnantonnixx; 27th April 2012 at 10:21 AM.

  2. #2
    ricki's Avatar
    Join Date
    Jul 2005
    Location
    uk
    Posts
    1,475
    Thank Post
    20
    Thanked 164 Times in 157 Posts
    Rep Power
    52
    HI

    I have an idea could you create a record in dns for each of you proxy servers under the name of proxy. Then change the ie proxy settings in group policy to the proxy name.

    When the computers pick up the ip address for the proxy it will get a round robin and all the proxy servers will be used,

    Richard

  3. #3

    jinnantonnixx's Avatar
    Join Date
    Mar 2011
    Location
    In the Calamatorium.
    Posts
    1,997
    Thank Post
    115
    Thanked 508 Times in 343 Posts
    Blog Entries
    2
    Rep Power
    286
    Thanks.

    I don't really want round robin though. I read the draft RFC for CARP and it mentioned a fairly sophisticated load-balancing which is what I want. Apart from filtering, the servers must cache and it would be best if the URLs were directed to the 'correct' server to make the most of the caching.

    The documentation is frustrating.



    What I'd like:

    Front end Squid (with fail-back caching in case a peer went south) > Carp hashes the URL > Passes to the appropriate peer server based on the URL > Server returns cache hit or fetches from the internet > happy bunny.

    I've got a cluster of Squid servers, but the distribution is based on a crummy algorithm in the pac file.

  4. #4


    tom_newton's Avatar
    Join Date
    Sep 2006
    Location
    Leeds
    Posts
    4,475
    Thank Post
    867
    Thanked 850 Times in 672 Posts
    Rep Power
    196
    Why are you clustering? Resilience or Load? Or both?
    I think you are trying to do something ICP-ish, where a cache hit on one member is nearly as good as a direct cache hit. That's great, but doesn't answer your load balancing problem.

    CARP is for redundancy, usually between 2 servers. For linux purposes I think the equivalent is VRRP.

    For load balancing I would suggest using a "proper" load balancer. The ones at loadbalancer.org are inexpensive and VERY good. You could roll your own but you would need CARP/VRRP between them for failover and to use something like haproxy to do the actual LB. Sounds like a lot of work, but your call

  5. Thanks to tom_newton from:

    jinnantonnixx (27th April 2012)

  6. #5

    jinnantonnixx's Avatar
    Join Date
    Mar 2011
    Location
    In the Calamatorium.
    Posts
    1,997
    Thank Post
    115
    Thanked 508 Times in 343 Posts
    Blog Entries
    2
    Rep Power
    286
    Thanks, Tom.

    Everything I read about Carp says it's the best thing in the world for load balancing. Well, perhaps not quite that, but they do drift towards that impression.

    I will check out you link for load balancing - thanks.
    Last edited by jinnantonnixx; 27th April 2012 at 02:00 PM.

  7. #6

    jinnantonnixx's Avatar
    Join Date
    Mar 2011
    Location
    In the Calamatorium.
    Posts
    1,997
    Thank Post
    115
    Thanked 508 Times in 343 Posts
    Blog Entries
    2
    Rep Power
    286
    Quote Originally Posted by tom_newton View Post
    Why are you clustering? Resilience or Load? Or both?
    I think you are trying to do something ICP-ish, where a cache hit on one member is nearly as good as a direct cache hit. That's great, but doesn't answer your load balancing problem.

    CARP is for redundancy, usually between 2 servers. For linux purposes I think the equivalent is VRRP.

    For load balancing I would suggest using a "proper" load balancer. The ones at loadbalancer.org are inexpensive and VERY good. You could roll your own but you would need CARP/VRRP between them for failover and to use something like haproxy to do the actual LB. Sounds like a lot of work, but your call
    Hold on - are you talking about the Squid CARP? The Cache Array Routing Protocol (CARP)
    This is the one I meant. I know there is a different CARP, but this CARP is definitely for load balancing against a hash of the URL.

    Every time I restart my front-facing Squid server with the "cache_peer <blah blah > carp" directive my Kerberos authentication goes west. It's as if the ticket has expired. The other peer servers work correctly on their own, and as part of an array from a pac file.

    I could just write something in Java to achieve the same thing from the PAC file, as I can just copy a hashing algorithm off the net.
    Last edited by jinnantonnixx; 27th April 2012 at 02:22 PM.

  8. #7

    Geoff's Avatar
    Join Date
    Jun 2005
    Location
    Fylde, Lancs, UK.
    Posts
    11,803
    Thank Post
    110
    Thanked 583 Times in 504 Posts
    Blog Entries
    1
    Rep Power
    224
    Quote Originally Posted by jinnantonnixx View Post
    Every time I restart my front-facing Squid server with the "cache_peer <blah blah > carp" directive my Kerberos authentication goes west.
    You can't do that. Kerberos auth is not compatible with CARP load balancing. You need to have two squids back to back. You can accomplish this by changing your peer lines to parents.
    Last edited by Geoff; 27th April 2012 at 02:48 PM.

  9. #8


    tom_newton's Avatar
    Join Date
    Sep 2006
    Location
    Leeds
    Posts
    4,475
    Thank Post
    867
    Thanked 850 Times in 672 Posts
    Rep Power
    196
    Ah, I had a different CARP You will still have to VRRP your "front-end" squids (for resilience) which then "carp off" to the backend peers (for LB), and even then you're going to get no more than a single-squid in terms of performance (unless your backend squids are doing something else too, like content filtering), so if it is scaling under load you're after I would still reccomend loadbalancer.org.

  10. #9


    tom_newton's Avatar
    Join Date
    Sep 2006
    Location
    Leeds
    Posts
    4,475
    Thank Post
    867
    Thanked 850 Times in 672 Posts
    Rep Power
    196
    Quote Originally Posted by Geoff View Post
    You can't do that. Kerberos auth is not compatible with CARP load balancing. You need to have two squids back to back. One to handle the auth then another to talk to your CARP peers.
    Yes, Kerberos and loadbalancing is difficult - we think we may have it cracked, but we're not holding our breath just yet.

  11. #10

    Geoff's Avatar
    Join Date
    Jun 2005
    Location
    Fylde, Lancs, UK.
    Posts
    11,803
    Thank Post
    110
    Thanked 583 Times in 504 Posts
    Blog Entries
    1
    Rep Power
    224
    Indeed, if you have more money than time just buy a load balancer. The squid peers in your pool can still swap notes via ICP/HTCP/CARP or whatever.
    Last edited by Geoff; 27th April 2012 at 02:52 PM.

  12. #11

    jinnantonnixx's Avatar
    Join Date
    Mar 2011
    Location
    In the Calamatorium.
    Posts
    1,997
    Thank Post
    115
    Thanked 508 Times in 343 Posts
    Blog Entries
    2
    Rep Power
    286
    Thanks guys. My home-brew solution is not looking too promising. I'm running filtering, proxying (of course) and caching on all my Squid servers.

    It's more complex still as I have NTLM and basic as failover.

    So is the consensus that if I'm using Kerberos then CARP won't work in Squid, even as parent?
    Last edited by jinnantonnixx; 27th April 2012 at 03:11 PM.

  13. #12


    tom_newton's Avatar
    Join Date
    Sep 2006
    Location
    Leeds
    Posts
    4,475
    Thank Post
    867
    Thanked 850 Times in 672 Posts
    Rep Power
    196
    Kerberos, in general, uses machine accounts, and as such needs to talk to the same *machine* all the time.
    You might do FrontSquid (auth,no filtering) Backsquid(no auth,filtering), and have a pair of VRRP Frontsquids talking to 2 or 3 CARP backsquids.

  14. Thanks to tom_newton from:

    jinnantonnixx (27th April 2012)

  15. #13

    Geoff's Avatar
    Join Date
    Jun 2005
    Location
    Fylde, Lancs, UK.
    Posts
    11,803
    Thank Post
    110
    Thanked 583 Times in 504 Posts
    Blog Entries
    1
    Rep Power
    224
    Try parenting, it might work.

  16. Thanks to Geoff from:

    jinnantonnixx (27th April 2012)

  17. #14

    jinnantonnixx's Avatar
    Join Date
    Mar 2011
    Location
    In the Calamatorium.
    Posts
    1,997
    Thank Post
    115
    Thanked 508 Times in 343 Posts
    Blog Entries
    2
    Rep Power
    286
    Does anyone have a working squid.conf file they can post which uses CARP with parent peers?

    TIA.

  18. #15
    ricki's Avatar
    Join Date
    Jul 2005
    Location
    uk
    Posts
    1,475
    Thank Post
    20
    Thanked 164 Times in 157 Posts
    Rep Power
    52
    HI

    I know this will be a stupid question but why do you need all this load balancing. Could you not just set each year group to a different proxy to balance the load. I have only see load balancing in county where full load balancing is needed when they are filtering for lots of schools.

    Richard

SHARE:
+ Post New Thread
Page 1 of 2 12 LastLast

Similar Threads

  1. Setting up load balancing on server 2003
    By Kyle in forum Windows
    Replies: 5
    Last Post: 25th November 2009, 08:48 AM
  2. Replies: 2
    Last Post: 31st October 2007, 02:25 PM
  3. Load Balancing
    By Jonny_sims in forum Thin Client and Virtual Machines
    Replies: 10
    Last Post: 6th December 2006, 09:43 AM
  4. Start Menus home folder load balancing.
    By Teth in forum How do you do....it?
    Replies: 6
    Last Post: 1st November 2006, 04:49 PM
  5. load balancing on print queues
    By browolf in forum How do you do....it?
    Replies: 3
    Last Post: 5th September 2005, 12:26 PM

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •