+ Post New Thread
Results 1 to 15 of 15
Internet Related/Filtering/Firewall Thread, Performance Monitoring Tool in Technical; We are having a lot of problems with time taken for web pages to load (users say things like "browser ...
  1. #1


    Join Date
    May 2009
    Posts
    3,118
    Thank Post
    273
    Thanked 836 Times in 628 Posts
    Rep Power
    332

    Performance Monitoring Tool

    We are having a lot of problems with time taken for web pages to load (users say things like "browser crashed" but while that isn't true - we can see very slow times for some pages). We need to monitor loading times in some way which replicates what a browser does (in terms of multiple request/response). I'm looking at producing some script (perl/python/bash - probably one of the first two) which drives wget and records the time to complete. Any other ideas for techniques/tools or potential problems in using wget?

  2. #2

    Dos_Box's Avatar
    Join Date
    Jun 2005
    Location
    Preston, Lancashire
    Posts
    10,467
    Thank Post
    607
    Thanked 2,193 Times in 1,006 Posts
    Blog Entries
    23
    Rep Power
    633
    Are these local (intranet) pages or from the web in general? Thgere was a web page benchmarking site published here just recently which should be able to help you out.

  3. #3


    Join Date
    May 2009
    Posts
    3,118
    Thank Post
    273
    Thanked 836 Times in 628 Posts
    Rep Power
    332
    Quote Originally Posted by Dos_Box View Post
    Are these local (intranet) pages or from the web in general? Thgere was a web page benchmarking site published here just recently which should be able to help you out.
    It's internet pages that are slow, pages from servers on the local network are generally fine.

  4. #4

    bossman's Avatar
    Join Date
    Nov 2005
    Location
    England
    Posts
    3,942
    Thank Post
    1,199
    Thanked 1,069 Times in 760 Posts
    Rep Power
    330
    @pcstru:

    One would presume that your school has the LA ISP for its internet connection?
    Then it would most probably be their upstream proxy cache servers which may be adding to the increase in slow web-page loading.
    They often don't get reset so they tend to lock up which causes a bottleneck for all the other schools who use those services.

    We have had several accounts of this in the past with our LA ISP.

    Check out your proxy cache first to make sure your proxy is not at fault.

  5. Thanks to bossman from:

    pcstru (31st March 2011)

  6. #5


    Join Date
    May 2009
    Posts
    3,118
    Thank Post
    273
    Thanked 836 Times in 628 Posts
    Rep Power
    332
    Quote Originally Posted by bossman View Post
    @pcstru:

    One would presume that your school has the LA ISP for its internet connection?
    Then it would most probably be their upstream proxy cache servers which may be adding to the increase in slow web-page loading.
    They often don't get reset so they tend to lock up which causes a bottleneck for all the other schools who use those services.

    We have had several accounts of this in the past with our LA ISP.

    Check out your proxy cache first to make sure your proxy is not at fault.
    We have checked as best we can. Our ISA server shows no particular issues and isn't stretched either processor or memory wise. We have one internal site which can hit the ISA server before coming back in and it's generally fine.

    Unfortunately the LA also say they have checked their end and there are no problems there. I need something to break the impasse or give us all a productive way forward.

  7. #6

    m25man's Avatar
    Join Date
    Oct 2005
    Location
    Romford, Essex
    Posts
    1,631
    Thank Post
    49
    Thanked 462 Times in 337 Posts
    Rep Power
    140
    Where does the Layer2 connection from your LEA ISP Router/Switch connect to your LAN?
    A simple topology diagram might help..
    Is the ISA in a two NIC Firewall mode or a single NIC sideways Proxy?

    In general the fewer switch hops between your gateway device and your core/distribution switch the better.

  8. Thanks to m25man from:

    pcstru (31st March 2011)

  9. #7


    Join Date
    May 2009
    Posts
    3,118
    Thank Post
    273
    Thanked 836 Times in 628 Posts
    Rep Power
    332
    Quote Originally Posted by m25man View Post
    Where does the Layer2 connection from your LEA ISP Router/Switch connect to your LAN?
    A simple topology diagram might help..
    Is the ISA in a two NIC Firewall mode or a single NIC sideways Proxy?
    Our ISA server operates 2 NIC's one is the internal proxy address for all internet traffic, the second connects into the LA provided box which connects the paper cups and string to the other paper cup and string at the LA end ...

    Response can be miserably slow during the school day. Edugeek used to be particularly bad - I think because of the multiple request/response (which is why I want whole page loads rather than ping etc) ... until ... I put it in the ... <gasp> white list. Now it's OK(ish). That suggested to me that the content filtering was a problem but the LA have just come back with some data which they say shows the response from their box to be OK (i.e. it's our problem not theirs).

  10. #8

    Join Date
    Mar 2011
    Location
    San Francisco, California
    Posts
    3
    Thank Post
    0
    Thanked 0 Times in 0 Posts
    Rep Power
    0
    <Removed unauthorized advertising>
    Last edited by Domino; 29th March 2011 at 10:08 AM.

  11. #9


    Join Date
    May 2009
    Posts
    3,118
    Thank Post
    273
    Thanked 836 Times in 628 Posts
    Rep Power
    332
    Quote Originally Posted by annieneustar View Post
    <Removed quoted unauthorized advertising>
    Thanks but I prefer an open public discussion for a reason...
    Last edited by vikpaw; 29th March 2011 at 10:18 AM. Reason: <Removed quoted unauthorized advertising>

  12. #10

    Join Date
    Mar 2011
    Location
    San Francisco, California
    Posts
    3
    Thank Post
    0
    Thanked 0 Times in 0 Posts
    Rep Power
    0
    We are having a lot of problems with time taken for web pages to load (users say things like "browser crashed" but while that isn't true - we can see very slow times for some pages). We need to monitor loading times in some way which replicates what a browser does (in terms of multiple request/response).

    I would suggest some third party tools that can be considered as best practices: perhaps external performance monitoring - which monitors your website from outside the firewall (through a true browser), that way you can identify errors, and at the same time, monitor the availability and performance of your website. Most importantly-- obtain an end users perspective (staging environment rather than production environment)

    In addition, load testing prior to launch is also considered a best practice where you are able to test capacity and id bottlenecks.

  13. #11


    Join Date
    May 2009
    Posts
    3,118
    Thank Post
    273
    Thanked 836 Times in 628 Posts
    Rep Power
    332
    Ah - sorry for not being entirely clear; the problem is not with our website but somewhere in the infrastructure between a users (student/staff) browser and the website they are visiting. So when a user accesses (say) BBC - Homepage, the page they want may take 10 seconds (or more!) to load or, when a class of 30 are directed by teacher to go to IANA &mdash; Example domains at about the same time, the site can take so long to load that people think their browser has crashed.

    For progress : I've set up nagios and am using check_http as well as some custom wget scripts. However, none of these is really capable of doing everything that a browser might do.

  14. #12

    Join Date
    Mar 2011
    Location
    San Francisco, California
    Posts
    3
    Thank Post
    0
    Thanked 0 Times in 0 Posts
    Rep Power
    0
    based on your information (without knowing your IT environment)...it can be quite a few things.... might be your DNS, CDN, bandwidth..but its sounds like Internal DNS...

  15. #13

    bossman's Avatar
    Join Date
    Nov 2005
    Location
    England
    Posts
    3,942
    Thank Post
    1,199
    Thanked 1,069 Times in 760 Posts
    Rep Power
    330
    @pcstru:

    Have you tried plugging a stand alone laptop directly into your external access route bypassing your ISA entirely. This would go directly to the internet and then you would be able to determine if it is your network LAN or the ISPs WAN.

    Then if you find out its your LAN you could do some diagnostics by hiring a fluke tester and having some fun.

  16. Thanks to bossman from:

    pcstru (31st March 2011)

  17. #14


    tom_newton's Avatar
    Join Date
    Sep 2006
    Location
    Leeds
    Posts
    4,475
    Thank Post
    866
    Thanked 850 Times in 672 Posts
    Rep Power
    196
    I use a few wget --mirror commands.. but that would be a bit sequential... threads in perl are messy, and LWP doesnt like threading.. so either perl, and fork wget commands, or python and its internal webget bits. nile_c might be able to give you a pointer there.

  18. Thanks to tom_newton from:

    pcstru (31st March 2011)

  19. #15


    Join Date
    May 2009
    Posts
    3,118
    Thank Post
    273
    Thanked 836 Times in 628 Posts
    Rep Power
    332
    Quote Originally Posted by bossman View Post
    @pcstru:

    Have you tried plugging a stand alone laptop directly into your external access route bypassing your ISA entirely. This would go directly to the internet and then you would be able to determine if it is your network LAN or the ISPs WAN.
    Excellent suggestion. Thanks. Did exactly that late friday and this mornings accumulated data clearly shows spikes and regular time outs on the LA side.

SHARE:
+ Post New Thread

Similar Threads

  1. Which network monitoring tool
    By Jobos in forum How do you do....it?
    Replies: 12
    Last Post: 7th September 2010, 09:59 AM
  2. Performance Monitoring Software
    By Hightower in forum How do you do....it?
    Replies: 8
    Last Post: 18th December 2008, 11:25 AM
  3. Advice on Monitoring tool
    By monkeyx in forum Wireless Networks
    Replies: 2
    Last Post: 25th June 2008, 08:35 PM
  4. Network monitoring tool
    By nicholab in forum How do you do....it?
    Replies: 11
    Last Post: 11th March 2008, 11:21 PM
  5. Replies: 1
    Last Post: 14th March 2006, 12:54 PM

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •