Internet Related/Filtering/Firewall Thread, Performance Monitoring Tool in Technical; We are having a lot of problems with time taken for web pages to load (users say things like "browser ...
28th March 2011, 02:56 PM #1
Performance Monitoring Tool
We are having a lot of problems with time taken for web pages to load (users say things like "browser crashed" but while that isn't true - we can see very slow times for some pages). We need to monitor loading times in some way which replicates what a browser does (in terms of multiple request/response). I'm looking at producing some script (perl/python/bash - probably one of the first two) which drives wget and records the time to complete. Any other ideas for techniques/tools or potential problems in using wget?
28th March 2011, 02:59 PM #2
Are these local (intranet) pages or from the web in general? Thgere was a web page benchmarking site published here just recently which should be able to help you out.
28th March 2011, 03:03 PM #3
It's internet pages that are slow, pages from servers on the local network are generally fine.
Originally Posted by Dos_Box
28th March 2011, 03:09 PM #4
One would presume that your school has the LA ISP for its internet connection?
Then it would most probably be their upstream proxy cache servers which may be adding to the increase in slow web-page loading.
They often don't get reset so they tend to lock up which causes a bottleneck for all the other schools who use those services.
We have had several accounts of this in the past with our LA ISP.
Check out your proxy cache first to make sure your proxy is not at fault.
28th March 2011, 03:23 PM #5
We have checked as best we can. Our ISA server shows no particular issues and isn't stretched either processor or memory wise. We have one internal site which can hit the ISA server before coming back in and it's generally fine.
Originally Posted by bossman
Unfortunately the LA also say they have checked their end and there are no problems there. I need something to break the impasse or give us all a productive way forward.
28th March 2011, 03:46 PM #6
Where does the Layer2 connection from your LEA ISP Router/Switch connect to your LAN?
A simple topology diagram might help..
Is the ISA in a two NIC Firewall mode or a single NIC sideways Proxy?
In general the fewer switch hops between your gateway device and your core/distribution switch the better.
28th March 2011, 04:33 PM #7
Our ISA server operates 2 NIC's one is the internal proxy address for all internet traffic, the second connects into the LA provided box which connects the paper cups and string to the other paper cup and string at the LA end ...
Originally Posted by m25man
Response can be miserably slow during the school day. Edugeek used to be particularly bad - I think because of the multiple request/response (which is why I want whole page loads rather than ping etc) ... until ... I put it in the ... <gasp> white list. Now it's OK(ish). That suggested to me that the content filtering was a problem but the LA have just come back with some data which they say shows the response from their box to be OK (i.e. it's our problem not theirs).
28th March 2011, 06:17 PM #8
- Rep Power
<Removed unauthorized advertising>
Last edited by Domino; 29th March 2011 at 11:08 AM.
29th March 2011, 09:58 AM #9
Thanks but I prefer an open public discussion for a reason...
Originally Posted by annieneustar
Last edited by vikpaw; 29th March 2011 at 11:18 AM.
Reason: <Removed quoted unauthorized advertising>
29th March 2011, 08:45 PM #10
- Rep Power
We are having a lot of problems with time taken for web pages to load (users say things like "browser crashed" but while that isn't true - we can see very slow times for some pages). We need to monitor loading times in some way which replicates what a browser does (in terms of multiple request/response).
I would suggest some third party tools that can be considered as best practices: perhaps external performance monitoring - which monitors your website from outside the firewall (through a true browser), that way you can identify errors, and at the same time, monitor the availability and performance of your website. Most importantly-- obtain an end users perspective (staging environment rather than production environment)
In addition, load testing prior to launch is also considered a best practice where you are able to test capacity and id bottlenecks.
30th March 2011, 09:52 AM #11
Ah - sorry for not being entirely clear; the problem is not with our website but somewhere in the infrastructure between a users (student/staff) browser and the website they are visiting. So when a user accesses (say) BBC - Homepage, the page they want may take 10 seconds (or more!) to load or, when a class of 30 are directed by teacher to go to IANA — Example domains at about the same time, the site can take so long to load that people think their browser has crashed.
For progress : I've set up nagios and am using check_http as well as some custom wget scripts. However, none of these is really capable of doing everything that a browser might do.
30th March 2011, 06:33 PM #12
- Rep Power
based on your information (without knowing your IT environment)...it can be quite a few things.... might be your DNS, CDN, bandwidth..but its sounds like Internal DNS...
30th March 2011, 08:48 PM #13
Have you tried plugging a stand alone laptop directly into your external access route bypassing your ISA entirely. This would go directly to the internet and then you would be able to determine if it is your network LAN or the ISPs WAN.
Then if you find out its your LAN you could do some diagnostics by hiring a fluke tester and having some fun.
30th March 2011, 09:08 PM #14
I use a few wget --mirror commands.. but that would be a bit sequential... threads in perl are messy, and LWP doesnt like threading.. so either perl, and fork wget commands, or python and its internal webget bits. nile_c might be able to give you a pointer there.
Thanks to tom_newton from:
4th April 2011, 09:05 AM #15
Excellent suggestion. Thanks. Did exactly that late friday and this mornings accumulated data clearly shows spikes and regular time outs on the LA side.
Originally Posted by bossman
By Jobos in forum How do you do....it?
Last Post: 7th September 2010, 10:59 AM
By Hightower in forum How do you do....it?
Last Post: 18th December 2008, 12:25 PM
By monkeyx in forum Wireless Networks
Last Post: 25th June 2008, 09:35 PM
By nicholab in forum How do you do....it?
Last Post: 12th March 2008, 12:21 AM
By disinfo in forum Windows
Last Post: 14th March 2006, 01:54 PM
Users Browsing this Thread
There are currently 1 users browsing this thread. (0 members and 1 guests)