Schools Broadband - Anyone else having Internet issues today?

synaesthesia · November 27, 2017

Ouch

Edu-IT · November 27, 2017

Thanks Dave.

SchoolsBroadband · November 27, 2017

Actually Talk Talk ADSL & FTTC circuits appear not to be affected. Just Talk Talk leased lines and EoFTTC's.

BT Wholesale, Virgin, Vodafone and other carriers are not affected too.

Dave

SchoolsBroadband · November 27, 2017

All circuits are now coming back online.

Dave

nathan · November 27, 2017

To be fair it was the people who are actually using their products who are complaining.

Clearly there are a fair few schools who are unhappy and seem to be experiencing very poor service.

Others have only commented when the excuses coming from SB are poor.

It's interesting that they're blaming shared filtering yet some customers are fine.

What's interesting about that?

They have shared clusters for filtering, if if a certain cluster is having an issue and you are not on it then you are fine.

This clearly indicates the bottleneck is the filtering and SB have already done work on fixing the issues.

Primus · November 27, 2017

What's interesting about that?

They have shared clusters for filtering, if if a certain cluster is having an issue and you are not on it then you are fine.

This clearly indicates the bottleneck is the filtering and SB have already done work on fixing the issues.

It's interesting because they clearly haven't got their load balanced properly otherwise everyone or no one would have issues - the fact that some people are ok is interesting.

The bottleneck is in a system designed and implemented by SB that is clearly not fit for purpose.

What's also interesting is that @synaesthesia has been defending SB but then has admitted that when he rang schools they they weren't surprised there was an issue with their broadband - doesn't that say it all?

nathan · November 27, 2017

It's interesting because they clearly haven't got their load balanced properly otherwise everyone or no one would have issues - the fact that some people are ok is interesting.

The bottleneck is in a system designed and implemented by SB that is clearly not fit for purpose.

What's also interesting is that @synaesthesia has been defending SB but then has admitted that when he rang schools they they weren't surprised there was an issue with their broadband - doesn't that say it all?

Or they have it setup to the suppliers recommendations and yet there are still issues which would be a supplier fault.

What's also interesting is that @synaesthesia has been defending SB but then has admitted that when he rang schools they they weren't surprised there was an issue with their broadband - doesn't that say it all?

eh? We all know there are known problems as do SB and synaesthesia, doesn't mean he can't defend them.

I, as many others on here just don't understand your position on consistently arguing that SB are rubbish when you only see a small portion of the story on here and aren't privy to the conversations going on in the background.

Primus · November 27, 2017

Or they have it setup to the suppliers recommendations and yet there are still issues which would be a supplier fault.

eh? We all know there are known problems as do SB and synaesthesia, doesn't mean he can't defend them.

I, as many others on here just don't understand your position on consistently arguing that SB are rubbish when you only see a small portion of the story on here and aren't privy to the conversations going on in the background.

But whether it's a supplier fault or not the fact is that you pay SB for a service that is not currently being delivered and has had problems for quite some time - how they procure their services isn't your concern beyond the fact that it works. It's unfortunate they've picked something that doesn't work right but that's their problem and their responsibility.

It seems fairly clear that if you're one of the schools affected you're having a huge lack of bandwidth on a regular basis and conversations in the background don't really matter since it's still occurring.

It's odd that he'd defend a service that his schools are clearly frustrated with - if my ISP kept letting me down like this there's no chance I'd be defending them and I'd definitely be exploring the legal options as I know some schools have in order to extricate myself from such a situation.

This is tax payers money that this company is receiving and the education of children that they are affecting by their inability to provide a stable service, whether or not I have their service in my schools I may well know people unfortunate to have purchased their solution and I'm also a tax payer - funds that we all contribute are going to a company that has dropped the ball pretty spectacularly. I may also have children of my own in a school where SB provide connectivity so to suggest I cannot take part in this discussion is ridiculous.

synaesthesia · November 27, 2017

Exactly this. I have sat down with many of the top brass at @SchoolsBroadband quite a few times, sometimes to discuss contracts, sometimes to discuss problems, sometimes because we wanted to touch base. I've said time and time again I will be fair, there's no point sitting here swearing because nothing's going to magically work because of a believe of inadequacy. I like them and have often defended them, yet I've also moaned about them. It didn't come a surprise to any of the schools I look after which does indeed say a lot. I know, the schools know and no doubt SB know that means contract renewal time in a few months is going to be tough for SB and will certainly result in the loss of business from at least one of them.

I have sat down with each of these schools and explained where I believe the weakness is, after SB sat down with me and explained the same. I therefore hope they will be as fair as I think I am.

The biggest issue is clearly filtering. If I go back over the last 3 or 4 years, from the top of my head there have been DDoS attacks, core failures, capacity issues and filtering related issues. Filtering capacity issues only came about with Lightspeed, and as all of our schools sit on the same cluster, they all invariably get affected at the same time. Prior to Lightspeed we had Fortinet filtering which pretty much just worked but wasn't quite granular enough.

DDoS attacks are moot; you'll suffer with them whether you're a small startup or Google. Core failures; I can only remember one from the top of my head which was a Fortinet issue, and Forti's engineers got that resolved in the end, if memory serves me that was a firmware issue

It appears as if the filtering issue is being looked at currently, and I don't need to talk about that; @SchoolsBroadband have said enough and will continue to update when they can.

The only bugbear, which I know, my colleagues know and my schools know is the capacity issue which often comes up about windows update time.

Yes there are issues, they are often overstated and exaggerated but that's bound to happen when they a) deal mainly with schools b) therefore a vast amount of those customers are here and c) they have such an active presence here also.

So, will I continue to defend them? When deserved, yes. Am I happy with them currently? No. I defend them because certain people are entirely unreasonable, have nothing to do with it and butt in just to try and make a point, and because sometimes the criticism is unfair. Not always.

Very few people are as upfront as they have been, and I've always admired their candour. I've also grimaced when one moment they are lauding the benefits of Lightspeed and the next moment they're swearing at them as much as we are, but promises get broken often in business and unfortunately we get the brunt of it.

Additional comment. If you are that concerned over the lack of internet in a school you may have child in, then your priorities would appear be in the wrong place.

Edited November 27, 2017 by synaesthesia

Primus · November 27, 2017

Additional comment. If you are that concerned over the lack of internet in a school you may have child in, then your priorities would appear be in the wrong place.

So if I'm concerned that a school is suffering from frequent connectivity issues my priorities are in the wrong place?

Depending on how the school teaches (eg. using G Suite or Office 365) frequent connectivity issues could seriously hinder progress.

Don't we want schools to make use of technology where appropriate? Do you think teachers will make use of technology that is very unreliable?

What about schools using SIP etc - connectivity issues can be even more serious for them.

Can I not be concerned that a company is taking tax payers money and not delivering an appropriate service?

Edited November 27, 2017 by Primus

SchoolsBroadband · November 27, 2017

Thanks @synaesthesia for being reasonable and defending us.

Perhaps we can all calm down a bit though.

Yes we've had problems for SOME customers which has been down to our filtering platform but the vast majority have not.

Yes we are running our Lightspeed clusters as recommended from Lightspeed from when they were installed.

Yes our core network has had 100% uptime since it's inception and the only noticeable Fortigate issues we've ever had is when Fortinet told us to upgrade to a different software version to fix a memory leak which then had some bad consequences before they then created an emergency patch for the patch they told us to upgrade to (and before anyone says we did test this but the bug wouldn't appear until 2-3 days of under load of over 3Gbits per box. These issues are now about a year ago and have not appeared since.

Except for an NNI issue with a carrier (as of this morning) all of our Direct Internet Access customers have never had an outage or slow speeds because of our network.

We have oodles of upstream capacity.

We are working on solutions to any problem we ever see on our network or suppliers kit with more to come.

We really do care about our customers. It pains me every time we get a complaint or someone isn't happen because we want to give great service where people can recommend us. We are not a huge corporation making millions of pounds profit a year. We re-invest the bulk in infrastructure and employ people to look after our customers.

Finally I should also say we've done a lot of great work with law enforcement agencies who have successfully caught and prosecuted offenders which have kept kids safe. If the schools would have continued to use their old content filtering solution this would likely not have happened. I do not go around bragging about this as its a very sensitive issue and the detail of it must be kept secret but what I can say is keeping kids safe is our NUMBER 1 priority and why schools use us over normal commercial ISP's.

We will continue to invest and strive to improve. We are deeply sorry when we let a minority of customers down and will do everything we can to make it back up to them.

Please can you we stop the back and forth (whether defending us or slating us) as we'll just continue to go round in circles with different opinions. As an when we have updates on things we are doing or any further issues then I will post information here as I did do this morning.

Thanks

Dave

nathan · November 27, 2017

So if I'm concerned that a school is suffering from frequent connectivity issues my priorities are in the wrong place?

Depending on how the school teaches (eg. using G Suite or Office 365) frequent connectivity issues could seriously hinder progress.

Don't we want schools to make use of technology where appropriate? Do you think teaches will make use of technology that is very unreliable?

What about schools using SIP etc - connectivity issues can be even more serious for them.

Can I not be concerned that a company is taking tax payers money and not delivering an appropriate service?

You assume alot.

It's not frequently for us.

"depending....." another wishy washy comment

Do you not think teachers should be able to think on their feet or are we going down this path of teachers being robots. They rely on far more than just an Internet connection.

Again, shows you haven't a clue about the situation. Our Sip trunks bypass the filtering so are not affected

Other than comments on an Internet forum, how on earth do you know about the service they offer if you don't use them?

If you were a customer of theirs your comments would carry far more weight, but you are not and comes across like you have an alternative motive.

If you are concerned about public money i would look at IT support suppliers who are charging primary schools upwards of £350 a day.

SchoolsBroadband · November 27, 2017

So if I'm concerned that a school is suffering from frequent connectivity issues my priorities are in the wrong place?

Depending on how the school teaches (eg. using G Suite or Office 365) frequent connectivity issues could seriously hinder progress.

Don't we want schools to make use of technology where appropriate? Do you think teaches will make use of technology that is very unreliable?

What about schools using SIP etc - connectivity issues can be even more serious for them.

Can I not be concerned that a company is taking tax payers money and not delivering an appropriate service?

oh and our SIP goes via a dedicated VLAN over our network into our carriers which doesn't even hit the content filtering and firewall solutions so continues to work even if other bits fail

Dave

synaesthesia · November 27, 2017

Regardless of who the provider is, change takes time, contracts aren't drawn up in a day and legal issues are no doubt a minefield.

For the same reason I will call any school that has to close because of no internet access "absolutely inadequate", relying so strongly on an internet connection that you couldn't possibly progress a child's education without it is a problem in itself. It may just be the way of the world these days, perhaps I'm being naive to think schools can exist without a connection for 30 minutes or that everyone just sits on their hands and does absolutely nothing whilst they wait... One of my schools is so dependent on IT and the internet that a downtime is a welcome break for nearly everyone where, and they've planned well enough that it in no way impacts T&L.

edit: Noted, @SchoolsBroadband - I'm out

Edited November 27, 2017 by synaesthesia

nathan · November 27, 2017

Thanks @synaesthesia for being reasonable and defending us.

Perhaps we can all calm down a bit though.

Yes we've had problems for SOME customers which has been down to our filtering platform but the vast majority have not.

Yes we are running our Lightspeed clusters as recommended from Lightspeed from when they were installed.

Yes our core network has had 100% uptime since it's inception and the only noticeable Fortigate issues we've ever had is when Fortinet told us to upgrade to a different software version to fix a memory leak which then had some bad consequences before they then created an emergency patch for the patch they told us to upgrade to (and before anyone says we did test this but the bug wouldn't appear until 2-3 days of under load of over 3Gbits per box. These issues are now about a year ago and have not appeared since.

Except for an NNI issue with a carrier (as of this morning) all of our Direct Internet Access customers have never had an outage or slow speeds because of our network.

We have oodles of upstream capacity.

We are working on solutions to any problem we ever see on our network or suppliers kit with more to come.

We really do care about our customers. It pains me every time we get a complaint or someone isn't happen because we want to give great service where people can recommend us. We are not a huge corporation making millions of pounds profit a year. We re-invest the bulk in infrastructure and employ people to look after our customers.

Finally I should also say we've done a lot of great work with law enforcement agencies who have successfully caught and prosecuted offenders which have kept kids safe. If the schools would have continued to use their old content filtering solution this would likely not have happened. I do not go around bragging about this as its a very sensitive issue and the detail of it must be kept secret but what I can say is keeping kids safe is our NUMBER 1 priority and why schools use us over normal commercial ISP's.

We will continue to invest and strive to improve. We are deeply sorry when we let a minority of customers down and will do everything we can to make it back up to them.

Please can you we stop the back and forth (whether defending us or slating us) as we'll just continue to go round in circles with different opinions. As an when we have updates on things we are doing or any further issues then I will post information here as I did do this morning.

Thanks

Dave

Good post and I'll agree to stop posting on the topic.

I know it's self centered, but all I care about are my schools and I know there are changes afoot to rectify the few issues we've received

KevByrne · November 28, 2017

I received an email from Schools Broadband last night (extract below), this seems to suggest that they are sticking with Lightspeed and not moving over to a different provider?

I am happy as I have been asking for this upgrade for quite some time as Lightspeed version 3 supports transparent SSL filtering.

>>>

Advisory - Lightspeed Systems Cluster Upgrades

Talk Straight / Schools Broadband Network Operations Centre (NOC) are committed to making regular network changes in accordance with network usage to offer our customers the best possible service.

We have been working towards upgrading our Lightspeed Systems rocket platform from version 2 to version 3 over the last few months. This major release brings with it increased reliability along with some of its latest features.

As version 2 support is coming to an end we will be upgrading all of our clusters over the Christmas / New Year period when impact can be kept to a minimum. We will thoroughly test the platform prior to your return in the new year to ensure this transition runs as smoothly as possible

Change Freeze

We will be implementing a change freeze window over the Christmas holidays for Lightspeed Systems modifications. We recommend that any non-urgent changes are left until after the upgrade as this may cause them to revert.

fiza · November 28, 2017

Maybe Dave @SchoolsBroadband can shed some light on this ?

SchoolsBroadband · November 28, 2017

I most certainly can.

Lightspeed are dropping support of v2.x as of the end of the year so we have to upgrade our clusters to the latest version if we want the platform to continue to be supported (which we obviously do).

Dave

KevByrne · November 30, 2017

We have today received email confirmation that Schools Broadband are moving over to Netsweeper.

fiza · November 30, 2017

We have today received email confirmation that Schools Broadband are moving over to Netsweeper.

[ATTACH=CONFIG]46479[/ATTACH]

Not sent to everyone me thinks!

Edu-IT · November 30, 2017

Haven’t got it either? @SchoolsBroadband

SchoolsBroadband · November 30, 2017

Haven’t got it either? @SchoolsBroadband

Can you check your spam? Do you get normal updates? If not then pm me your email and Ill check. We have / will send different emails to partners than direct customers.

Dave

fiza · November 30, 2017

I normally get the status updates from Carole but nothing about netsweeper as yet.

Edu-IT · November 30, 2017

Can you check your spam? Do you get normal updates? If not then pm me your email and Ill check. We have / will send different emails to partners than direct customers.

Dave

No, nothing, and we normally get service emails. PM Sent.

KevByrne · December 1, 2017

We have been having issues since yesterday with Google blocking our web searches due to unusual traffic from "our" network, this only appears to be affecting our Windows 7 Machines.

Within Internet explorer we receive an unlimited number of reCaptcha messages so cannot continue, within Chrome the reCaptcha is slightly different and does work after a minute or so of trying.

The IP address specified within the error message is for Filter 17 so it's not necessarily our site that is causing the problem, as such I was just wondering if anyone else was having this issue (particularly on Filter 17).

This has been logged with SB however I was just wondering is anyone had any ideas on why this would only be a Windows 7 Issue and of any possible work around?

Schools Broadband - Anyone else having Internet issues today?

Recommended Posts

synaesthesia

Edu-IT