notalot Posted December 8, 2015 Posted December 8, 2015 Hi, sorry if this is in the wrong place please feel free to move it. We have just finished setting up a Squid3 server on Debian. It works fine 99% of the time but some sites it refuses to load at all, YouTube happens to be one of them. When you browse to http://www.youtube.com the page just sits loading, nothing comes up on screen (I've left it for 30 minutes with nothing loading) and it never times out. We have the same problem if we try to go to Google (even though google is the homepage) it loads google for the first time then doesn't do it again. I have no idea what the problem is with it as I'm still very new to Squid, we have tried a few things. Turning off the Squid Cache Adding the DNS_4_first setting to the config Setting debian to use IPv4 first. Checking that the file descriptors are within limits Looking in the Access.log file the connection request doesn't show up. See below a copy of our config (the cache has been turned back on as it doesn't appear to fix the problem). #Recommended minimum configuration: shutdown_lifetime 2 seconds cache_peer UPSTREAMFILTER parent 80 0 no-query no-digest never_direct allow all acl localnet src 10.60.148.0/22 acl SSL_ports port 443 563 1863 5190 5222 5050 6667 acl Safe_ports port 80 # http acl Safe_ports port 21 # ftp acl Safe_ports port 443 # https acl Safe_ports port 70 # gopher acl Safe_ports port 210 # wais acl Safe_ports port 1025-65535 # unregistered ports acl Safe_ports port 280 # http-mgmt acl Safe_ports port 488 # gss-http acl Safe_ports port 591 # filemaker acl Safe_ports port 777 # multiling http acl CONNECT method CONNECT http_access deny to_localhost icp_access deny all htcp_access deny all http_port 8080 hierarchy_stoplist cgi-bin ? access_log /var/log/squid3/access.log squid #Suggested default: refresh_pattern ^ftp: 1440 20% 10080 refresh_pattern ^gopher: 1440 0% 1440 refresh_pattern -i (/cgi-bin/|\?) 0 0% 0 refresh_pattern . 0 20% 4320 # Leave coredumps in the first cache dir coredump_dir /var/spool/squid3 ############################################################################## ##Auth Settings ############################################################################## ### negotiate kerberos and ntlm authentication auth_param negotiate program /usr/local/bin/negotiate_wrapper -d --ntlm /usr/bin/ntlm_auth --diagnostics --helper-protocol=squid-2.5-ntlmssp --domain=DOMAIN --kerberos /usr/lib/squid3/negotiate_kerberos_auth -d -s GSS_C_NO_NAME auth_param negotiate children 10 auth_param negotiate keep_alive off ### pure ntlm authentication auth_param ntlm program /usr/bin/ntlm_auth --diagnostics --helper-protocol=squid-2.5-ntlmssp --domain=DOMAIN auth_param ntlm children 10 auth_param ntlm keep_alive off ### provide basic authentication via ldap for clients not authenticated via kerberos/ntlm auth_param basic program /usr/lib/squid3/ext_ldap_group_acl -R -b "dc=DOMAIN,dc=local" -D squid@DOMAIN -W /etc/squid3/ldappass.txt -f sAMAccountName=%s -h dc01.DOMAIN auth_param basic children 100 auth_param basic realm Internet Proxy auth_param basic credentialsttl 10 minute ### acl for proxy auth and ldap authorizations acl auth proxy_auth REQUIRED # aclname acltype typename activedirectorygroup acl allowedsites dstdomain "/etc/squid3/allowedsites.txt" acl blockedsites dstdomain "/etc/squid3/blockedsites.txt" acl exceptedsites dstdomain "/etc/squid3/exceptedsites.txt" acl prioritysites dstdomain "/etc/squid3/prioritysites.txt" ### http_access rules # allow unrestricted access to prioritysites http_access allow prioritysites # enforce authentication, order of rules is important for authorization levels http_access deny !auth # prevent access to basic auth prompt for BlockedAccess users http_access allow allowedsites http_access deny blockedsites # Access http_access allow localnet # block all other access http_access deny all I would appreciate any help in getting to the bottom of what's causing this.
psydii Posted December 8, 2015 Posted December 8, 2015 Try using the developer tools in the browser to identify what elements in particular are taking a long time to load. 1
jinnantonnixx Posted December 8, 2015 Posted December 8, 2015 (edited) I haven't worked with Squid for a while, so I'm pretty rusty. Is there a reason why you're using Kerberos and NTLM? Kerberos is by far the preferred protocol. Even though your config offers Kerberos before NTML, it is the client browser that decides on the protocol that it ultimately uses. Squid exceptions are logged in the file 'cache.log'. This command will list the last events logged by squid. The -f option keeps on polling the file, showing you a live view until you press Ctrl-C to escape. tail -f /var/log/squid/cache.log Have you configured the squid stats tool? This is a CGI script which should be installed by default, but not necessarily configured. First find the file locate cachemgr.cgi Copy the file to this location: /var/www/cgi-bin/cachemgr.cgi Start a browser and connect to http://your_squid_server/cgi-bin/cachemgr.cgi There's a whole heap of useful goodies in here. The following is a paste from my notes - it must have seemed important to me at the time. The most useful menu items are: NTLM User Authenticator Stats This shows the array of NTLM authenticators and their status. If they are busy, they will show the current requests. Pay attention to the 'Queue Length' field. If there are requests backing up, they will show as a non-zero number. This indicates that the server is too busy. Negotiate Authenticator Statistics This shows the array of Kerberos authenticators and their status. 5 Minute Average of Counters This shows a comprehensive range of statistics. Of particular interest: client_http.requests metric. Regular inspection of these values tells us what a normal load is, but as a rule of thumb, we can regard 150 requests/sec as being on the heavy side. client_http.all_median_svc_time. This metric shows how long it took to complete a single HTTP transaction. A good value is around 0.01 seconds. dns.median_svc_time. This is the measure of time taken to resolve DNS cache misses. A good value is under 0.05 seconds. High values indicate a problem with the primary DNS server (or network). How's your DNS server - is it stressed out? If you can, you might want to try using Google's public DNS servers just to see if this has any effect. You'll need to restart the Squid service, so do this out-of-hours. In the squid.conf, add dns_nameserver 8.8.8.8 8.8.4.4 Edited December 8, 2015 by jinnantonnixx 1
notalot Posted December 9, 2015 Author Posted December 9, 2015 Thanks to both of you, Psydill - The developer menu doesn't show anything just that the request was started. jinnantonnixx - Thas a massive help. We were using a config that was from a guide and we haven't thinned it down yet just wanted to get things working then look at improving them, as it stands this is in development currently and not live. From your information it turns out that there is a DNS related issue with Google and YouTube but not with any other site I tried. Its a little odd but I would guess it has something to do with the LA filtering forcing safesearch. If I run the DNS on the server through 8.8.8.8 then it works fine but bypasses some of the filtering which I really don't want to do. Now its just a case of working out how to resolve the issue.
jinnantonnixx Posted December 9, 2015 Posted December 9, 2015 You might want to try these settings in the squid.conf. negative_ttl 1 minute positive_dns_ttl 1 hour negative_dns_ttl 1 minute These set negative hits to have a short life time, so that if something is wrong with your DNS, and a negative hit is encountered (by error), then the negative hit isn't remembered for long. squid : negative_dns_ttl configuration directive
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now