EXA_Mark

It already has pushed them down consistently for the past twenty years since DSL was first introduced. The very first leased line i was involved within installing for ourselves as a business in the 90's (not Exa) was a 256Kbps line at £18500 a year! You have to bear in mind in comparing FTTP (OR) to Leased Lines or DarkLight that those services are congested and do have contention and speeds do vary (sometimes massively), and sometimes to specific connections (such as MS, Google or Apple) on update or patch days. Leased Lines or DarkLight (Dark Fibre) should not, as long as your ISP has enough capacity in their core.

Just a quick word of warning to all about Openreach FTTP, although I am sure many of you are aware of this. From a technical perspective, all fibre to the premises (Leased Lines, Dark Fibre and such) are FTTP. However FTTP as talked about a product replacement for DSL/EoFTTC/FTTC/Gfast from Openreach uses GPON technology, and is an asymmetrical service, meaning the download is faster than the upload (how Openreach DSL type products have worked since 2000, not using GPON mind, but asymmetrical). Dark Fibre services, such as DarkLight or Leased Lines are symmetrical, meaning you have the same speed up and down. Also, FTTP (Openreach) is not a guaranteed speed service, it is subject to peak time congestion and slow down, (when Peak time actually is now that half the people are working from home all the time I am not sure though). It is in the terms and conditions of OR and others such as Virgin broadband services. The biggest thing to take out of this though is the limits of the upload on OR FTTP services, on 300Mbps FTTP you reference the upload is only 50Mbps. If you can get the 1Gbps one it is just over 200Mbps. Now you might say that is more than sufficient, but remember the quote at the start of this thread, more teachers and students are using cloud service than ever before, and that means sending data back out to the web MUCH more than when you are simply web browsing or streaming. In the past few years, we have seen upload speeds increase on our networks four times faster than downloads speed. Now do not get me wrong, people use the download element much more, but a few years ago it was a ratio of about 10:1, so if you had a 100Mbps pipe we would see peak uploads of around 10Mbps. It is now closers to 45% at peak. Meaning if you had a 100Mbps line we would see peak uploads of around 45Mbps. Bear than in mind. That is now. It will continue to increase. So if you are thinking an upload speed 50Mbps will be ample for us for years to come, it probably won't it almost isn't now. And just to finish, please don't misunderstand me, FTTP is a huge jump compared to current mass available technologies such as DSL and FTTC, and it is a service we provide ourselves too, but if you are on a 1Gbps leased line and seriously considering this as an option then do get your upload stats checked first. And this is not us trying to push more expensive or lucrative products, the actual margins on most connectivity services are very similar, it is about having something that works.

Unfortunately, it is the old adage, Steve, it depends where you are. For instance, if you are in one of our DarkLight areas we install 10Gbps bearers on our Dark Fibre and these come in under £6K a year with only £500 install, with 300Mbps on 10Gbps coming in at £3,500. For 1Gbps on 1Gbps, at the moment though you are generally seeing this in the region of £6k-£9K per year. I'm not surprised on the current pricing you have on your 300Mbps line (I assume it was a three-year contract signed in 2018) as the wholesale tail on Gigabit circuits (especially those provided into ISPs directly through Openreach has come down massively in the past three years. The price being added on for the actual internet bandwidth (not included from Openreach) has remained fairly constant for the last few years though, and whilst it has come down, it is the tails that have made the biggest impact. If you'd like me to send you over a quote just PM me your details, and don't worry, we are not a pushy sales type company, we send you a quote and then are there if you have questions or want to proceed.

Hi, You are right we are not great. We are awesome Seriously though I do not know who has said what, and that is not for me to say their opinion is wrong, even if it is. We take what we do very seriously, and have done since we started supplying schools with connectivity and filtering 17 years ago. There are plenty of people on here (and thanks to those who have said so in this thread) that like what we do, and our customer retention rate of over 95% would seem to back that up. Utlimately, do we have issues from time to time. Sure. In fact every provider on here has threads (some many more than others) on their outages or issues. But if we do have them, we try and make sure we communicate, and learn from them if we do. At present we are one of the largest non-grid for learning ISPs in the UK, and serve thousands of customers for connectivity and filtering and thousands more for our Exa Foundation. We have been very fortunate to win many awards (not the sort you sponsor and get one), including ISPA (our equivalent of BAFTA) for Best Business ISP, Best Customer Service and Best Broadband multiple times. In fact we have had over 60 finalist nominations in 16 years from ISPA. And we are shortlisted for Five next month, including Best Business ISP. As an aside to that we also write our own software in house, such as SurfProtect, so we tend to be amongst the very first in the world to fix problems such as Google or Captcha updates, as we are not waiting for update patches to push across consumer equipment in the same way or for third parties to write fixes, we do it ourselves. And our ExaBGP service is used by many of the worlds largest tech companies, including Microsoft, Twitter, Facebook, BT and the BBC. So are we perfect? No, but we are having a bloody good go at getting there. Let us put you in touch with ANY of our customers (not just testimonial ones) or others that have moved from competitors you are considering, and let them speak for us. Hope you give us a go.

Hi, Going through one of our partners or resellers does not mean you can't contact our support team. Give them a call on 0345 1451234 or email [email protected] and they will help you.

You are absolutely correct. Whilst we can issue IPv6 to any customer on our connectivity when it is something you want to run through SurfProtect Quantum, it adds in MUCH more complexity around configuring profiles, analytics, diagnosing route paths and such, which is why at present we don't. It is not to do with network routing, but rather specific requirements for the service. However, IF you would like to work with us directly, we will develop this, working with you, to get all the bits in place. We could start work on this in Q1/Q2 next year, depending on your availability. PM me with your contact details if you are interested and I'll pass it onto the team and they'll be in touch and go through what would be involved.

Just a quickie from a suppliers perspective. We have been running dual-stack across our network infrastructure on IPv4 & v6 since 2007 and a huge amount of network traffic coming back into us, in particular sites such as Google and Facebook by default is via IPv6 nowadays. BUT very few customers across any sector, Education included have made the move, but the rapid deployment and exhaustion of v4 across the world (not us specifically) will mean people will have to do at some point in the not too distant future. If any of our customers on here want an IPv6 allocation, just give your account manager a call/email, or PM me. It should go without saying, but I will in case anyone asks, we do not charge for this, it is part of the standard service.

Thanks @cdCache @TJ-Diggers if you need anything from us or have any questions at all please let me know. If you like to speak to any of our schools, local to you or further afield again just let me know. We've been providing internet connectivity and filtering to thousands of schools since 2003, so plenty for you to speak to, schools who've been with us for 15+ years or others who have just joined. And it is not just connectivity, take a look at http://www.exa.foundation to see what else we provide to schools (Foundation stuff is all free of charge, and available nationwide). Enough sales pitching, I try not to do that on here, it always feels awkward, PM me if you need anything else.

Hi @Clansman, Firstly welcome to Edugeek and thank you for the post, as you can tell it is something we feel very strongly about. We are developing some courses (for free) as part of our Exa Foundation, http://www.exa.foundation where each year we run hundreds of events for schools throughout the UK, if you'd like to get involved, or chat with me/Exa on anything we can jointly do to educate, please PM me or give me a ring on 0345 1451234

We are putting together a top tips guide just for this very reason Peter, as soon as it is done, I'll let you have it, and post it on here too. Probably late next week.

Part 2 of 2 One of the other posts on the thread commented about how they’d been on a local authority connection previously and were down for two weeks due to an attack. This is not that uncommon, unfortunately. Those sorts of providers typically simply do not have the type of resources, people, experience, different data centre, peering and therefore they often simply have to let the attack run its course. Another big issue is dealing with an attack that you block out of your network, but it still hits the upstream before your edge network, which no “Lamborghini” is going to stop. The major peering points in the UK do not have DDOS hardware or solutions installed. We also have DDOS protection service from our upstreams, and again this helps mitigate it away from us and our customers quickly, but again, only if it the traffic comes this way, and you have to bear in mind that even when our upstream providers block it, it is still hitting somewhere! So unless you decide to push all your traffic through an external source first (which in itself can cause other issues such as latency) such as Cloudflare (which works for a website but not for an internet connection), then you will have that issue, as you would with private peering/interconnects and many other issues. And even those type of companies only promises to mitigate “most attacks”. So again, no one simple solution. What you need to have is many different solutions, and we are looking at even more, and ability to write your own to meet specific requirements, and experience, which fortunately we have in our engineering and R&D side, but passing down the answers from those firefighters in the heat of an attack to the support team (and bear in mind how many different sorts of attacks they can be, its not simply a case of training), is not always easy, and so sometimes a mixed message gets out. For which I am really sorry. We are looking at ways of improving how we can get this out better in the future, without slowing down those sorting the issues. We also try and put out the updates on status.exa.net.uk but on one occasion the message was not put out quick enough, simply, the people who normally do the updates were not in the office, and it got missed until one of our customers pointed it out. One of the other challenges is the sheer volume of traffic that networks such as Google and Amazon have going to and from their networks. Simply for cost reasons, they will always prefer peering, or private peering, to transit. So if you have an attack and try and re-route one of those, often their own internal automated systems may/will override your mitigation attempts and still send it down a path you don’t want it to use. I am very aware this even though I said this is a long post, it is now a VERY long post, so I will just try to end it by saying a couple of final things. To those who were affected by the DDOS indirectly, our please accept our apologies, we always try to do better, even if it is not us causing the problems. To everyone who expressed their understanding of our dilemma, thank you. When the teams are under pressure, it is always really appreciated to hear and read these posts. Yesterday we implemented an update internally which has allowed for even faster detection and mitigation today. We had another attack this morning on a different school (who has never been attacked before). From start to end it was mitigated it in under two minutes. There will, of course, be others in the future that take us longer I am positive, but be assured we are also going to be working and implementing more solutions, internal and external to reduce the impact as much as possible. Finally, let me say that Exa has a zero-tolerance policy on DDOS, and all these attacks will be reported to the appropriate authorities. A DDOS is a criminal offence, and we will treat it, and act on it as one. We believe ISP, The Government and the schools can play a massive part in educating people and make it clear that this sort of criminal behaviour will simply not be tolerated. We hope that should a DDOS ever happens to your school, no matter who your service provider is, you will work with them to identify the individual and make sure they and the rest of the school know that you will not tolerate it either. If anyone has any other questions, please do feel free to ring me or PM me.

OK, let me start off by saying, this is going to be a long, pretty technically detailed post. I feel I cannot give a short answer to the many questions. It is so long in fact, that I have gone over Edugeeks 15000 character limit, so this thread post 1 of 2. Let me start by giving a few facts as there seems to be some misinformation on here, or lack of clarity, or maybe even confusion that we may have caused. We had four separate days (not entire days) of noticeable very large DDOS attacks, this has not been going on for months. The first one was last week, then three others this week (I'll come back to this later in the post). Over the past few days, the attacks have been against three specific schools (not SME or Corporate customers). Two each for two of them, and one for another. The schools are not connected from an area or Academy trust perspective, but we believe the attacks on the three schools are related, probably by individuals within the schools communicating via forums or chat rooms. One of those individuals initiating one of the DDOS one has already been personally identified by the school and has been dealt with very severely. A DDOS attack is a criminal offence, and we treat them as such when we can identify the individuals. The other two individuals are still being sought and we are working directly with the schools to do so. Where we manage the firewall and filtering, it is often quite easy for us to follow the breadcrumb trails, you'd be surprised how big a trail people leave, and find the perpetrators which were the case, with the first school, Two other schools were using their own in house firewalls and filtering solutions, and as such we are supporting them and doing all we can to help them discover the identities of the students performing the DDOS. So onto some facts and figures. On a normal school day, we see peak traffic around the network of between 12Gbps for a normal day & 20Gbps for a busy one peak traffic across our network. All of our core Data Centres /PoPs connect at up to 100Gbps. So you maximum throughput on any one port can only be 100Gbps. We have three upstream providers (for what is referred to within the industry as commercial transit) from our DCs, with more than 4 x our overall network peak traffic of available throughput/transit. Your average internet service provider may have 1.5 times peak (so very little overhead)! On top of that, we have multiple 10Gbps peering sessions, including a direct PNI (Private Interconnect) with Google (not public peering). As the vast majority of traffic in the UK goes through peering this means we currently have more than 9 x our peak throughput on our edge and we can increase this when we need to While this may seem excessive traffic flows can change and therefore some headroom is required to not face surprises. Any customer going through our network would be able to traceroute to different locations and see the different routes. For instance, try google, and you’ll go through the PNI, try BBC and you’ll go through peering and try BT and you’d most likely through transit. We can also prioritise how traffic leaves our network if we needed to for maintenance or emergency reasons (such as DDOS) and in some measure affect how it comes in. We publish our peering points on https://www.peeringdb.com/net/524 If you are interested in this you can also look at our competitors to see what they have in comparison. It does not, however, show private interconnects (as peeringdb is to help you find peers). It also shows you things such as our average traffic levels too. This information is however self-published and should, therefore, should be taken with a pinch of salt. On the specifics over the 4 DDOS days (not concurrent full days). The first attack took place on the 26th September at around 8 am, a status update was put on the status.exa.net.uk page within a few minutes of the attack commencing. It was targeted against a single school (the one that we have identified the individual since, and there has been no recurrence). The attack was substantial and saturated nearly all of the peering points. Our engineers' saw this on our mitigation platform some change happened to improve the situation in less than ten minutes, but not stop entirely. By 08:20 we had cleared all DDOS traffic from transit but peering points we still saturated (I’ll come back to this later on). By 08:25 this was limited down to specifically the peering points in London, LONAP and LINX (the biggest exchange point in the UK). All traffic that was going through transit or to Google was now not affected. Then at 09:25, the attackers started using Google for the attack. Google was the largest source of all traffic, which meant the PNI to Google was then affected, but all other traffic was then fine. At 09:45 Google changed how the traffic was reaching us due to the attack. Google has an automated system to optimise its traffic. We, therefore, had to perform some network configuration changes to protect our peering links. However, the traffic was lower so customers were not seeing the same level of impact. This continued to reduce down to zero over the coming hours with our mitigation services in place. By 2 pm there was no sign of any DDOS activity on the network. The next series of DDOS attacks happened on the 2nd October starting around 1:25. This was fully mitigated within five minutes. About 15 minutes later a different school (the third targeted) this was fully mitigated within a few minutes And then yesterday (4th October) the same school (the second one) was targeted again. This time the attack came through our transit, not peering locations. This meant over 80% of traffic was going through as normal, so the likes of Google and BBC were mostly unaffected, but services with some VoIP providers (as some mentioned in the thread) which do not have peering with us at the exchange points or privately, were affected. This time the traffic was identified as coming Amazon’s cloud services. For very obvious reasons we cannot simply block all of AWS traffic, as they provide cloud-based hosting. So we had to mitigate in a different way, which unfortunately ended up taking a few hours, although there were improvements to many sites/locations within a few minutes of it starting. So that is a quick(!) summary of the four separate worth of attacks. Sometimes at overlapping times. It is always much harder to deal with this sort of thing when they are coming from lots of different routes to different customers. When our customers are under a DDOS which is affecting more than the school, our operational practice is to first restore service, and then get back the affected school back online. We knew all of the attacks had commenced within the schools, or from someone who had been in the schools, as following an attack we change the IP of the connection. When following this IP change, a new DDOS occurs again the school, it is generally down to someone from the school looking up the IP address of the firewall or gateway. It could also be a compromised machine, but we are still to see this. Generally speaking, with 20 years experience of this, it is nearly always a kid. I will now try and answer some of the other comments or concerns. @GTX and a few others kind of said similar, “A network that big needs DDOS protection. It's not cheap but it the world we live in now”. I absolutely agree. Which is why we do have DDOS protection in place and have done for over 15 years; I’ll cover @SchoolsBroadband “Lamborghini” reference later on specifically. We have different DDOS protection within our network, some inside, some using our suppliers. Unfortunately, despite what some others might want you to believe, there is no single box solution, that will cover every eventuality. One of the bits that were mentioned was ExaBGP. The software we wrote in house, that to be fair is used by many large technology companies, for different reasons, including DDOS mitigation. Most DDOS ISP mitigation services use FlowSpec to stop traffic from a particular protocol for or to an IP (for instance all DNS going to the IP of a school), whilst keeping the school up. In a lot of cases, this alone is enough to end an attack. The main issue is what do you do when an attack is bigger than your upstream. No equipment within your own network can then mitigate the issue. It must be coordinated using your upstream network. This problem is made worse as there is little check from the sender that the traffic they are sending should come from their network. Industry efforts exist to attempt to sort this issue but it is not going to be an overnight thing. https://www.manrs.org/isps/guide/antispoofing/ So to mitigate an attack, you have to have many different solutions, which to an outsider, looks like one solution to one problem when it is many different ones, badged under the same umbrella. The D in DDOS is for Distributed (Denial of Service), therefore there is more than once source of the attack to handle. There are other things we have inside our network, like Dave, and these all help with mitigation. And all you can do is mitigate. You cannot stop someone attacking a network, no matter what devices or solutions you have in, what you have to do is mitigate the impact as quickly as possible. The reality is we have attacks on a regular basis, as do most ISPs but the majority of them are invisible to everyone other than the person/organisation that is targeted when we can not “simply” mitigate the issue in a way invisible to everyone. If you google for “ExaBGP FlowSpec filetype:pdf” you will see many industry-leading experts talking about it, and how they integrate it. A few open-source and commercial products are using it. As I do not expect you to just take my word for it, here are a few references Cisco speaks of it: https://www.ciscolive.com/c/dam/r/ciscolive/emea/docs/2019/pdf/BRKSPG-3012.pdf Juniper: https://conference.apnic.net/data/41/apricot-ddos-mitigation-using-flowspec_1456208439.pdf T-Mobile: https://ripe74.ripe.net/presentations/93-20170512-ripe74-flowspec-interop.pdf We work within the Internet Engineering Task Force (IETF) to make sure ExaBGP is a good as it can be. You can google: "ExaBGP implementation site:tools.ietf.org" to check. Several commercial DDOS companies are using ExaBGP too, so when @SchoolsBroadband said he’d “heard it was good” it is perhaps a bit of an understatement but ExaBGP is not a panacea. But it is just one piece of a very complicated issue. Which is why major companies such as Twitter, Microsoft and Facebook have had services taken down by DDOS attacks. If it was just a case of throwing unlimited money at a problem to fix all variants then those companies would do that. It is not a simple thing at all. Part 2 (due to character length limit) is directly below this post.

Could you PM me your contact details so I can get our support team to look into this. There is nothing happening on our network anywhere now that should be causing this. It is often easy to assume it is one thing (such as a DDOS) when it could be something completely unrelated and I want them to check for you. Thanks - - - Updated - - - Statement coming in the next couple of minutes. It has taken me far longer than I thought it would when I started writing it hours ago!

I guess Dave @SchoolsBroadband has a lot more spare time to be on the forums than I do I also tend not to post into other supplier-specific issues threads such as SB has had over the past week with their filtering or in the past with DDOS too, as quite simply, as someone else in this thread wrote, all suppliers will have an issue from time to time, it is how they handle them, and how they respond to them. Seriously though, I do try and keep an eye on the forums, but as you can all appreciate with your busy day jobs too, I don't always get to check them as often as I would like. The post was brought to my attention yesterday evening as I was out of the office yesterday most of the day, and I am writing a fairly detailed long post now, which will be up on here very shortly. So please don't think I am ignoring my fellow edugeekers.

There is no issue that is affecting this from our side, so if I can ask you to give our support team a call on 0345 1451234 they will be able to take a look at this for you, or if you PM me your contact details I will get them to call you.

Hi @Alis_Klar, Firstly, thanks for moving across to Exa. Can I ask, are you with us directly as a customer or through one of our partners? I am asking from a support perspective. Please, can you give our team a call and we can go through anything with you, especially if you are seeing issues on the Root CA. Just give us a call on 0345 1451234, or PM me with your details and I'll get them to call you. Mark

Thank you for the comment, and I just want to say, any outage for us is unacceptable even ten minutes. A fix was identified for this yesterday, which is for a very obscure bug, that hadn't raised its head before, and this was only on SurfProtect Fusion, so Quantum, Proxy and Cloud were unaffected. The fix is being pushed out tonight. Obviously, there is for any company (and I have commented on it on this thread before), a chance that as you grow and are successful you encounter problems or challenges you've not seen before. That's why we have spent so much additional time and resources on SurfProtect recently to make sure we are in a position where this doesn't affect our customers. As I have said to others, I'm really sorry for any inconvenience, we will always strive to do better, at everything we do.

Sorry about the short outage yesterday, which was just under ten minutes, which is still about ten minutes too much. This only affected customers on SurfProtect Fusion, and we do try and get any issues on the Status page asap. We know what caused the fusion issue yesterday and a fix will be pushed out tonight to make sure this can't happen again.

Thanks for the comment. If you have signed up to our status notifications you will get notifications on anything SurfProtect related, however, the brief problem yesterday only affected SurfProtect Fusion, so if you are on Cloud, Proxy or Quantum, you would not have been impacted. I also suspect the Google issue mentioned by the other person, was specific to that school as we had not other support tickets or calls about it before the issue yesterday or since. We are going to update the status notification system in the future, so you can just have notifications on the variant you have, ie Quantum, as opposed to getting something that may not be related or affect you.

I am really sorry you were impacted yesterday morning, on this. Only customers on SurfProtect Fusion were affected, so I am going to assume you are on this. If you would like to move over to SurfProtect Quantum, just PM me and I'll get your account manager to get in touch and sort it out. The outage on Fusion yesterday was a total of 6 and a half minutes, followed very shortly after by 3 minutes. The root cause of this was identified and a fix is being pushed out today. You may have also seen a very short outage this morning (again Fusion only), this was human error and was resolved in a couple of minutes. Once again, really sorry for any disruption this may have caused. One thing that is a puzzle though is you mentioned about Google Verification problems. This was not a symptom of yesterdays problem, and we had no other reports before or since. Can you give our support team a call on 0345 1451234 and they'll have a chat with you about it. Problems on Google verification can be caused by numerous issues, but primarily it is the IP or IP range that the request is coming from that Google has the biggest issue with. On Fusion, Google see you direct IP addresses, not a pooled or proxy range.

A quick update on this, which as you said was raised on Friday to our support team. Google has made changes on how they look at the pool of IP addresses coming from requesters, such as SurfProtect, and this has caused the solution that was in working on Recaptcha/verification, to have issues. This was passed to our SP Dev team and we expect a solution to be in place within 24 hours. Sorry for any inconvenience this is causing anyone, and we will have a fix in place for Googles changes very soon. I will keep you posted.

Sorry for the delayed reply on this, it has just been brought to my attention in the past hour, and I'd not been on the forum for a few days. Let me try and answer this, and a few other messages within this thread. But before I do that, let me firstly apologise to any of our customers who have not had the level of service you have come to expect from Exa, or we expect ourselves to deliver. For those customers who have had disruption with SurfProtect over the past couple of weeks I am deeply sorry for that. Before I get into this in depth, I just want to clarify a point that has been made a few times on here, the network at Exa has not been down, at all, the internet connectivity has remained up at all times (I am not saying individual customers have not had unrelated outages such as a fibre cut or an Openreach fault), but that the "network fault" mentioned is specifically relating to either SurfProtect or related DNS issues. With that out of the way, let me give you a (I apologise again), long answer. As many of you know or can appreciate, software development is never simple, especially when we are working on a living, growing system such as SurfProtect, where a bug or outage is immediately impactful. I wanted to give you a bit of an update about where we are with SurfProtect, what we have already done and are doing to make sure we have the stability back quickly, and what we are doing going forward. The last quarter of 2018 has seen a massive increase in customers using SurfProtect Quantum. With more than 500 new schools moving on the new Quantum platform. 2019 will see our team contacting existing SurfProtect Cloud customer and migrating them and offering to do the same for our Fusion customers who are still using the Stormshield firewalls. It is therefore not surprising to hear that the load on our infrastructure has more than doubled in around 2 months, and with the migration of SurfProtect Cloud customers, plus the usual Jan-April school moves, the expectation is that we will see the load a least quadruple before the end of Q1 2019. And we have to be ready for this, which is what has been causing some of the issues over the past couple of weeks. About half of the SurfProtect development team is busy adding further resilience and new features to the product, the others have been hard at work implementing a solution which will allow us to take such an increase in traffic. SurfProtect Quantum is currently filtering nearly 5 Gbps of traffic every day at peak time, with this traffic expected to reach over 10 Gbps before the end of Q1 and over 20 Gbps by the end of Q2. This substantial growth, while very great from a business perspective, is not without its challenges. Our engineers are currently working hard adding new servers our customers can use, to make sure we have enough capacity way ahead of time to make sure this growth can be painless. This work should complete for early January (prior to the schools' return from the Christmas break), with some servers already installed in Manchester and London. Also before the end of the year, our team will be able to provide per-customer Surfprotect AD and Proxy servers, in order to better spread the load across the SurfProtect infrastructure Without going into very technical details, our proxy customers are currently using `ad.quantum.exa-networks.co.uk` and `proxy.quantum.exa-networks.co.uk` to get their traffic filtered. The browsers are then connecting to these destinations to get the traffic filtered. In order to apply a divide and conquer approach to the issue, our engineers have developed a solution which allows us to provide different customers personalised answers. The way the internet works is by resolving these name to unique computer IP. This process uses a system called DNS (which many of you will be familiar with) and is similar to looking up a phone number in the yellow pages. The solution is quite similar to a busy call centre in say London, which to better help its customers open some new office in Bradford, and get the local directory (yellow pages) updated for all northern customer with the Bradford office number. In this analogy, we have to make sure that the issue of out-of-date copies of the yellow pages ringing the wrong number does not apply. That’s why the answer we give has to have a very short lifetime, to make sure the answer is always the right one not out of date. Since early this week, when your customers are asking for the location of the SurfProtect proxies, our backend systems have been able to direct them to the nearest and most capable machine to deal with the traffic. We are looking forward to using this feature to redirect customers with problematic applications toward some dedicated servers configured to help us diagnose the issue(s) without requiring any intervention from the customer themselves. Hopefully allowing for faster issue resolution. Our analytics database, held at its peak this month well over 100 billion records (we are inserting 10s of thousands of records per second during the day ), which all our customer can now search in real-time. Because of the huge growth in those logs one of our main projects for this last quarter and next is to work on changing some of our underlying technologies. This can occasionally cause some interruptions as we are deploying the new software or hardware to cope with this, and again, my sincerest apologies for this. We do our utmost to keep disruption to a minimum. HTTPS now count for over 2/3 of all connections. It continues to cause challenges for all filtering providers, especially when Google and now others, including CloudFlare which host around 10 million domains, are taking technical measure to actively prevent any form of traffic inspection, including for filtering. The team is therefore busy to pro-actively work on solving the challenge that future mass ESNI deployment will undoubtedly cause. https://blog.cloudflare.com/esni/ Unlike some filtering vendors who may decide to simply open cloud service providers such as Amazon Web Service, Microsoft Azure, CloudFlare or Google Cloud, SurfProtect identifies the actual service to make sure any accessed URL can be filtered logged and is available in the analytics for review. This is in line with requirement laid out by the government on Counter-Terrorism and the DfE but can cause massive headaches. Over the last year, as an example, these were some of the challenges we had to figure out a way around. • Microsoft Outlook relying on HTTPS connection failing to fall back to HTTP as was previously the case. • TLS key negotiation issue which was required to be able to change the key negotiation cypher for certain releases of Windows 10. • For security, many phone/tablet applications do verify the certificate authority used for signing the certificate, preventing HTTPS filtering! • Many applications not expecting HTTP filtering and using port 443 (SSL/HTTPS) for custom protocol, each of them requiring a unique and tailored solution, and even different version of the same product, such as Twitter works completely differently at a network level on an App to the web. While most of this work has been performed in stealth mode with little or no impact, it also tends to go unnoticed and can sometimes look like there is no development on SurfProtect. A statement that could not be further away from the truth. The development team continues to expand and we recently brought in new operations and project manager to oversee the deployment of the many new features and updates. The team is hard at work to make sure the new version of TLS (3.0) the protocol securing HTTPS connection can be analysed as some applications are now switching to only support TLS 3.0. As well working toward other new features which I hope to announce very soon. To those on the thread who had put some heart warming comments, thank you, I think many people forget that the developers are real people, who have all now seen this thread, and critiscm is hard to take sometimes for all of us, but it is important for people to be aware of it, and for them to see the positive comments is most welcome. We will continue to do our best to make sure that those who have not had the best experience recently, get to see the improvement quickly.

I am so tempted to get into this with you on here Dave, but perhaps not the best place So let's just agree to disagree! As you know, to take Dark Fibre and then be able to offer internet connectivity through it and light up the fibre, (hence DarkLight), you have to have a PoP/DC in each geographic location to terminate one end (the customer being the other) and then either breakout or connectivity back to another DC in your network to breakout. From what I understand of your network you do not have that many DCs? Of course my information might be out of date since we met earlier in the year. On the matter of the others doing Dark Fibre soon, I really do hope this is the case. Our network was not built to just use City Fibres DF, but any, and the kit we have in all of our DC's and PoP throughout the country, with another three opening this year, makes it all the better, where we have more options for getting fibre for other areas. It was a shame when BT (and others) blocked Openreaches decision to launch Dark Fibre last year. Although how competitive each of them end up being remains to be seen. But competition is good, and here's hoping we have more network suppliers making their infrastructure open to ISPs through Dark Fibre as opposed to just waves.

Just to be clear Dave, TTB, Virgin, BT, Openreach and Vodafone, do not sell Dark Fibre. They all sell a lit product (leased line or EAD type) or a wave length (Openreach). And currently are limited to up to 10Gbps circuits. We've already delivered 40Gbps and are about to do some 100Gbps services dedicated fibre, not wavelengths. So whilst the 100Mbps pricing has come down, the Gbps or 10Gbps is nowhere near our DarkLight pricing, from any other provider. I know this for certain as we have now done hundreds of Dark Fibre circuits and even more on traditional leased lines with all of the providers you mention (excluding Vodafone). 1Gbps 1:1 on the intial 10Gbps DarkLight service is less than £6,000 a year, none of the others are anywhere near that. Also, I believe SB would take the City Fibre product through Entanet (City Fibre Wholesale) and that is just lit and contended GPON product, not the Dark Fibre product we have CF install on our behalf, which as you are aware is a very different thing. Finally, there is no lit product available in West Sussex from CF/Enta, only Dark Fibre / DarkLight.

I can't really comment on your specific situation with City Fibre at that time, but if you want me send me a PM with the school address I can tell you for sure what the situation would be from our side. As far as sweating the older lines, I am sure in some areas over the past five or six years, they have reused the existing fibres, if they bought them, but I can tell you from first hand experience, the vast majority of work we have done with them in Bradford, Leeds, Wakefield, Calderdale, Sheffield, Milton Keynes, Northampton, Rotherham, Doncaster and other areas (including West Sussex) have been new digs and new fibre pulls, I know this first hand, as the other end of the splice is new into our DC's and PoPs in those areas. Our average dig is well over 150M per site, with some schools I can specifically link you to, including mentioned on here, have been over 800M. It all depends on the areas, what else is around and the type of construction required. I can say for certain if you had two schools within 150M of our network you would be getting DarkLight installed no problem.