Slow login when a whole class logs in over wifi

[Jollity]

My major project for half term is to work out why the laptop suites are taking so long to login when a whole class logs in at once. Login time for one laptop logging in on its own is fine, but when the whole class logs in at the same time a proportion of the laptops end up taking ridiculously long - 15+ minutes. Possibly not a wifi issue, but it seemed the most important factor so I put it in this forum.

 

When the Windows 7 laptops were new a year ago, login times for the whole class were quite slow but acceptable. The accounts use a mandatory profile and settings have not changed that much. The Ruckus wifi system needs patching up to date. I also want to look into some of the hotfixes for login delay issues, but the fact that it only really happens when the whole class is logging in implies to me there is some kind of overload issue. The slow logging in laptops do not always seem to get stuck on one stage of the login, everything just seems to take longer.

 

Has anyone any suggestions for things I could try or investigate?

[SYNACK]

How many laptops, how many waps in range and are the laptops and waps 802.11abgn so that the clients can be split out between waps and frequencies. Wireless is a shared medium and is always going to suck in comparison to cable, you need to do everything you can to spread the load out as otherwise your 300mbits of n coverage is going to be split between say 30 laptops all screaming at once and colliding causing wait states meaning the more you have the less of the maximum you can have to share between them, lets be optimistic and say you end up with 60% efficiency, that is 180/30=6mbit/s max without any overhead for anything else like encryption and frame headers.

 

The answer is more available bandwidth to share and more waps to share between.

[Jollity]

I just realised I forgot to mention how many clients - 20 laptops, with 2 N WAPs in range. The Ruckus system can do 5GHz as well as 2.4GHz on the same SSID, but I am pretty sure the laptops are 1x1 and 2.4GHz only (much to my annoyance). However the other suite of 20 apparently also have issues have 2x2 wifi cards and I think can use 5GHz.

 

Do you have a rule of thumb for access point to laptops ratio?

Edited October 20, 2013 by Jollity

[SYNACK]

Not really, a lot is down to the devices and environment, there was another thread like this a couple of weeks ago, check the wap graphs for the snr and amount of interference, it could be the environment causing lots of signal bounce due to high power. You could also look at upgrading the client wlan cards as most are just mini PCI-e.

[psydii]

In parallel to looking at the wifi, this post will help you identify any performance issues on the Windows side of things:

Slow Boot Slow Logon (SBSL), A Tool Called XPerf and Links You Need To Read - Ask Premier Field Engineering (PFE) Platforms - Site Home - TechNet Blogs

 

It may also be worth taking a look at RAMMAP on your fileserver. If the majority of your ram is not "mapped files" then you need to reboot your server. If after an hour of use in a busy environment "mapped files" are not dominating the graph then you need more ram in your server. <--I am skipping over the whys and wherefores, read Windows Internals, Sixth Edition, Chapters 10 and 11.

 

Roaming AppData can really hurt performance. Tools like xperf and procmon can help you pinpoint the bottlenecks. A good 'template' to follow if you want to use procmon is presented at The Case of the Veeerrry Slow Logons - Mark's Blog - Site Home - TechNet Blogs it may be easier to use that than xperf.

[neilmac]

1x1 in 2.4 Ghz will give you a maximum data rate of 65 Mbps (or 72.2 with short guard interval).

 

Remember that only one device will be able to transmit at once. In close in environments, there aren't usually collisions, but where there are there is an increase in backoff times, so you need to confirm if this a problem by looking at CRC error and retry stats. Depending on the data rates available to each device, you are going to have to split about 50 Mbps of throughput between many devices. Questions to consider:

 

What are the throughput requirements for the logon process ?

What encryption is being used ?

 

You are probably slowed by latency, if all devices are close to each other I wouldn't suspect too much corruption, more a question of congestion.

 

Can you find anything out from the logs to see if there are any timeouts ?

[Jollity]

Thank you all, some useful suggestions and reading for me.

 

 

there was another thread like this a couple of weeks ago

This one seems relevant.

 

You could also look at upgrading the client wlan cards as most are just mini PCI-e.

That is an interesting possibility. I had thought that it would be impractical due to a need to plumb a second aerial into the laptop. Is it actually more likely to be just a matter of swapping a card?

 

Remember that only one device will be able to transmit at once.

That is a useful confirmation. I was wondering whether if I have two 1x1 clients and a 2x2 access point the access point could talk to both at once - if I understand you right the answer to that is no. So 10 1x1 clients all trying to transmit will put the same load on an access point over a particular time period as 10 2x2 clients all trying to transmit, even though the 2x2 ones can send more data in that period?

 

In close in environments, there aren't usually collisions, but where there are there is an increase in backoff times

Could you expand on "close in environments": wouldn't rate of collisions have more to do with congestion than distance from access point? Or am I missing something?

 

What are the throughput requirements for the logon process ?

The total data sent and received during login is something I would like to measure. I was thinking I may be able to measure it with a windows performance counter. Any other suggestions?

[neilmac]

OK,let's see if |I can address these questions.

 

First, read this blog post from "Wirednot" :Are You a Good Wi-Fi Client? | wirednot

 

As SYNACK mentioned, and is often overlooked, client drivers may be important and the blog post refers to this and other issues I am about to mention.

 

So, let's see if I can easily explain a concept that is covereded in CWNA but not deeply analysed until CWAP.

 

In an environment, all devices are within earshot of each other as the rooms are small, so wherever a device is that transmits, when it transmits, all other devices nearby will hear the transmission.

 

There are 4 essential rules of WiFi. Rule 1 is "A device can receive or transmit, never both at the same time".

 

So, in your environment, a device gets to transmit, what do other devices do ? They listen for the PLCP header and look for the FRAME_LENGTH field, in order to know how long the frame will be in the air. After the PLCP header, sent at a lower mudulated rate, the MAC header is transmitted AT THE HIGHER MODULATION RATE. In an environment where everyone can hear everyone else, it's a good thing as the next value you need to look for is the DURATION_VALUE which tells you how long to wait AFTER the frame has been fully received before you can think about transmitting.

 

Now, if your devices are further apart, you may receive the PLCP header OK, but can't decode the higher modulation of the MAC+data portion of the frame, so you don't pick up the DURATION_VALUE. In this case, you will most likely receive a corrupted frame (fail CRC), which means you must wait even longer before getting to transmit (since you didn't read a duration value you wait extra long to ensure the frame - ack process completes), but you will still be quiet while a transmission is underway.

 

If devices are very spread out, they will not detect the PLCP header, or will, but weak enough to be ignored, in which case you will get more collisions as devices will transmit as if there is no other frame in the air.

 

A professional designer will take this into account when planning a network, as the biggest thing that kills network performance are frame retries, so we must balance everything between power, modulation rate, capacity and coverage. It's not easy to do at the high end of network planning. The CWDP study guide is 900 pages long and you have to already be CWNA first before you can become a CWDP.

 

This concept isn't easy to explain, and is a very difficult concept even for quite advanced engineers. It's even more difficult to pinpoint when in the field, however with the right training and the right tools, a WLAN engineer would be able to discover the reason for WLAN issues.

 

Now, to cover your other questions, 1x1, 2x2 just means the capability of the device transmitting and receiving, the higher the spec the higher the modulation rate that will be available on a per frame basis, but it's still always only 1 transmission at any one time.

 

This is why adding another access point on the same channel doesn't add capacity, as it's just another device that needs to be quiet if someone else is talking.

 

By the way, 802.11ac has something called Multi-User MIMO, but it's not in any wave 1 kit, and reading the specs I wouldn't be surprised if it's not implemented any time soon, if at all. This allows transmissions to multiple devices at once. Till then, rule 1 sticks.

 

(Refer to CWNP | Certifications for explanations of CWNA/CWAP/CWDP etc)

 

I hope this helps.

 

NM

Edited October 20, 2013 by neilmac

[SYNACK]

Most laptops had two antenna anyway and used them in the same way the early waps did using the one that got the best signal at the time, a newer card should manage both at the same time.

[Michael]

To me this clearly appears to be a bandwidth issue, rather than a wireless issue. Are your APs connecting at 100Mbps each or 1000Mbps?

 

Gigabit connectivity (even when using just wireless G) with many devices makes all the difference.

[Michael]

Another way to look at it (for example), is say all your notebooks are connecting at 65Mbps each. 65 x 20 = 1300Mbps (theoretical maximum), but I'd say (more than likely) you have two APs within range, so this would halve to 650Mbps each. Either way, this is considerably more than if you had two APs connecting at 100Mbps each.

 

I know my calculations are pretty basic, but it makes sense seeing as one or two notebooks probably logon quite quickly, but adding more slows everything down.

[Jollity]

Thank you for such a detailed answer Niel Mac, that was really illuminating. I was not aware that Wi-Fi devices were all listening to each other's packets in that way - certainly plenty to learn on Wi-Fi.

 

To me this clearly appears to be a bandwidth issue, rather than a wireless issue. Are your APs connecting at 100Mbps each or 1000Mbps?

I am pretty sure I checked and they were Gigabit, but I will double check.

[SYNACK]

Another way to look at it (for example), is say all your notebooks are connecting at 65Mbps each. 65 x 20 = 1300Mbps (theoretical maximum), but I'd say (more than likely) you have two APs within range, so this would halve to 650Mbps each. Either way, this is considerably more than if you had two APs connecting at 100Mbps each.

 

I know my calculations are pretty basic, but it makes sense seeing as one or two notebooks probably logon quite quickly, but adding more slows everything down.

 

Um, I'm pretty certain that unless your waps has a stack of radios (xirrus) you are limited to the max single rate. Under g that is 54mbit between all devices as that is the max you can fit in to the radio band available at that modulation. It shared so one device and wap have to share this, you add more devices and it drops like a stone especially with collisions. With n you can get 300mbits which easily overwhelms a poltry 100mbit link so gig is really needed to make it not pointless. This is also why the ac waps have two gig ports that can be teamed as a gig link could be overwhelmed.

[Tall_Paul]

Very interesting stuff. When our school was built (before my time) the SLT mandated that the majority of devices would be wireless (for convenience, hem, hem). As a result we have around 1200 wireless clients and logon times compared with the few wired machines we have always been one of the biggest issues.

[psydii]

If you've got 2x2 "n" radios in both client and ap *and* you have at least -70dB everywhere, *and* you force local profiles *and* you don't redirect appdata, and you aren't bridging broadcast or Mutitcast traffic, then 'wireless' logons should be <2 minutes at all times (i.e. the length of time it takes to build a profile from scratch on a laptop with a HDD), and browsing the web, access files across it should also be ok. It gets a bit messier if you are trying to access large files, or hundred of very small files.

 

As you mix up the clients and their use cases it also gets trickier.

 

With a Meru network (and 2x2 clients) we've been able to have a class full of laptops working on photoshop files stored on a server, where as a Trapeze network (with a lot of 1x1 clients) completely failed. In both cases the ratio was 1 AP for every other classroom. With a a/b/g Chantry network and XP we had less than 1 ap to every two rooms, and we had 2-4 minute logon times, but never tried to edit photoshop files.

 

If you need guaranteed higher bandwidth, for example by insisting on roaming or mandatory profiles you get a massive spike at logon (25MB+ * 30 Devices = 750MBytes) which in a reasonable case (150Mbits/s) gives you a theoretical (ie assuming no collisions and no other traffic and every client able to sustain 150Mbit/sec) of trafic that takes a minimum of 40 seconds to transfer, all you can do is try to increase the bandwidth in that location. Unless your coverage is very thin, the benefits of increasing overall AP density are quickly outweighed by the co-channel interference problem, however this is where the local environment comes into play - if your walls are reasonably radio opaque you could up your coverage to one dual radio ap per room and halve your transfer time for rooms that previously did not have an AP, which in turn would improve performance in the rooms with existing provision as the load was now on another channel. Once you've hit the signal strength/AP density sweet spot, You have to start adding additional channels into the same physical location with an enterprise wifi system that can load balance across them. You can then also restrict lower performance clients to specific channels/SSIDs/radios to keep them from reducing the performance of the critical ones.

Edited October 21, 2013 by psydii

[Michael]

Um, I'm pretty certain that unless your waps has a stack of radios (xirrus) you are limited to the max single rate. Under g that is 54mbit between all devices as that is the max you can fit in to the radio band available at that modulation. It shared so one device and wap have to share this, you add more devices and it drops like a stone especially with collisions. With n you can get 300mbits which easily overwhelms a poltry 100mbit link so gig is really needed to make it not pointless. This is also why the ac waps have two gig ports that can be teamed as a gig link could be overwhelmed.

 

I know about the differences between G and N, but I can see why you thought that way

 

That's interesting however that AC APs come with two gigabit ports. I still think it'll only be a matter of time before someone works out how to get 10Gbps over CAT5e, working to 100m.

[neilmac]

Remember that 300 Mbps is only the maximum data rate available in 5 GHz with 2 spatial streams with a 40MHz channel and short guard interval.

 

WLAN devices are only required to support up to MCS 7, so in 2.4 GHz that's a maximum data rate of 72 Mbps. With no frame aggregation, the maximum throughput you will get is 57 Mbps, and this needs to be shared between ALL devices. This is a theoretical maximum, once you factor in multiple devices the true throughput drops.

 

1 AP with 5 clients all modulating at MCS7 in 2.4 GHz gives a throughput of around 40 Mbps, about 6.7 per device.

 

1 AP with 10 clients, all at MCS 7 will get about 3 .7 Mbps per device.

 

And, this is with everyone modulating at the highest rate available, if stations have to rate shift down, then the picture gets worse.

 

There are tools to analyze this, when you plan your network before deployment these are all the sorts of scenarios you need to consider.

 

 

NM

Edited October 21, 2013 by neilmac

[Jollity]

The 300Mbps figures given for N wifi reminds me of the "up to" figures on ADSL lines.

 

It seems I have found the source of the problem rather more easily than I feared. Symantec Endpoint appears to be downloading 300MB of updates on each client, every time each one turns on, at least if it has not been turned on today already. So it is not really surprising connection speeds grinds to a halt. After waiting half an hour for Symantec to sort itself out, the laptops take about 40 seconds to turn on and another 40 to log in - not great, but acceptable.

 

Now to work out how to get Symantec to behave! I think I might have to get the laptops to turn on each night to update or something.

[neilmac]

And we conclude, the wifi was behaving as it was designed to ....

 

Great you got it worked out.

 

NM

[Jollity]

And we conclude, the wifi was behaving as it was designed to ....

Indeed...

 

Well I gained some useful insights on WiFi issues for the future.

[seawolf]

The 300Mbps figures given for N wifi reminds me of the "up to" figures on ADSL lines.

 

It seems I have found the source of the problem rather more easily than I feared. Symantec Endpoint appears to be downloading 300MB of updates on each client, every time each one turns on, at least if it has not been turned on today already. So it is not really surprising connection speeds grinds to a halt. After waiting half an hour for Symantec to sort itself out, the laptops take about 40 seconds to turn on and another 40 to log in - not great, but acceptable.

 

Now to work out how to get Symantec to behave! I think I might have to get the laptops to turn on each night to update or something.

 

The only way I've found to get Symantec to behave is to uninstall it. Good luck!

[featured_spectre]

Synack hit the point. Many wifi companies and brands are awesome for single user through to 10 user wifi but add more in or larger profiles etc things will suck. Ruckus has stepped forward on this and can be used to handle a large amount of users (provided profiles are reasonably small...ie arpund the 3mb mark).

 

Other brands of wifi (xirrus, aruba, meraki) are also stepping up now and I would be willing to bet that out of all, xirrus would be the next brand that says yes, one AP can service X range, with Y many users, and do it well.

 

In all my final advice is this...test every brand thoroughly. Have a class willing to help out. Install each brand individually, test the log ins and speeds of 30 machines at once...then rinse and repeat. Also another thing to factor...thickness of walls between AP and machine, any wire msh in the walls, other RF antenna causing havok or all APs on the same chanmel for example etc. Factor in everything and move forward.

 

Good luck

[neilmac]

Only 1 device can talk at once, so if your design is going to be critical you need to work from the very beginning with a good understanding of how WiFi works. The differences between vendors are not so great unless you know how to configure it to get the extra benefits.

 

Manufacturers will give figures but you have to understand how AP loading works, plus how to make extra gains with QoS, balancing, toggling data rates and other tweaks that will change the environment.

 

10 internet users will behave vastly different to 10 VoIP phones, simply adding more access points does not increase performance and can often make things worse. It's never about how many devices per access point.

 

You have to start with the client devices and work up from there. Complex network design needs a skilled professional, if you try and do it cheaply and with minimal skills then you design faults into the network that will be difficult to put right later. You should thoroughly analyse your client devices and their needs, this will tell you how many access points you have to install, then you have to try and balance the signals.

 

Vendor selection is about in the middle on a task list of things to do when planning a network.

 

Really, if it's an important network that will be used for more than just casual browsing, you need to plan correctly, and get professional help.

 

NM