NetworkDirector Posted October 22 Posted October 22 I understand that this PHP vulnerability makes services need to go down, but Class Charts need to get their act together, they can't just go offline whenever they want without telling anyone, schools and customers rely on them, I guess no behavior being logged today then. 1
gomalley Posted October 22 Posted October 22 2 minutes ago, Anyer said: I never get anything as the main admin for Class Charts, yet now I know the email I did a message trace and everyone in Senior Leadership got the email bar me! When it's all up and running I'm going to get them to add me to this notification email list. Wow! Good luck - I've asked many a time about being notified about issues and been no offering from them
Anyer Posted October 22 Posted October 22 Just now, gomalley said: Wow! Good luck - I've asked many a time about being notified about issues and been no offering from them What's even more bonkers is that their notification list includes staff members that no longer work here (which I can see from the message trace), but I can't be added on?! 1
Homer Posted October 22 Posted October 22 Smells like a very manual way of them having deployed their infrastructure for this length of an outage. I'm sure if they'd used up to date automation for deployment their infrastructure would have been patched and active again ages ago.
psydii Posted October 22 Posted October 22 1 hour ago, Homer said: Smells like a very manual way of them having deployed their infrastructure for this length of an outage. I'm sure if they'd used up to date automation for deployment their infrastructure would have been patched and active again ages ago. I'm wondering if it is worse than that, because even pre-devops sysadmin practices ought to have allowed them to patch quicker than this. Perhaps they have the technical debt of running on old frameworks that require production code to be re-worked before they will run on supported php versions? Wonde appears to be back up now. (Hadn't checked since 6am, so might even have been up by the start of the school day).
gomalley Posted October 22 Posted October 22 3 minutes ago, psydii said: I'm wondering if it is worse than that, because even pre-devops sysadmin practices ought to have allowed them to patch quicker than this. Perhaps they have the technical debt of running on old frameworks that require production code to be re-worked before they will run on supported php versions? If this is indeed the case then thank goodness there is only 1 more day till half term - and dear god hopefully that gives them enough time
psydii Posted October 22 Posted October 22 I have nothing to base that on, other than they are still down while others are back online.
gomalley Posted October 22 Posted October 22 I wouldn't say years (at a guess from your post count) of experience with multiple different vendors is nothing to base it on - it is a good hypothesis - just a shame we can't test the theory!
Homer Posted October 22 Posted October 22 1 hour ago, psydii said: I'm wondering if it is worse than that, because even pre-devops sysadmin practices ought to have allowed them to patch quicker than this. Perhaps they have the technical debt of running on old frameworks that require production code to be re-worked before they will run on supported php versions? Wonde appears to be back up now. (Hadn't checked since 6am, so might even have been up by the start of the school day). Ooooooh I bet you're on the money there. PHP 5.6 anyone? 😂
Anyer Posted October 22 Posted October 22 Had this forwarded on from one of our teachers that gets the notification emails: Quote Following our earlier email, we’re pleased to let you know that we’ve now restored a range of services, including Parents Meetings, Room Bookings and Tes Magazine. Thank you for your patience while we worked through the global issue affecting thousands of online services worldwide. This was due to a global issue with widely used internet technology that sits behind thousands of online services. In order to address this, we took some services temporarily offline as a precaution. Where things stand We’re bringing our services back online in stages, checking each one as we go. ClassCharts is expected to be available, ready for UK registration (23rd Oct). Some of our other products remain offline as a precaution while we finish our restoration work. What this means for you The services listed above are available and can be used as normal. ClassCharts: please check back later today. When it’s available, you can use it as normal. Other products remain offline for now. There’s nothing you need to do, but we are continuing to work on this to resume all services as soon as possible. Staying safe We’ll never ask for your password or login details by email. If an email looks suspicious, please contact https://help.tes.com/support/home What happens next We’ll email you again as soon as everything is restored. Thank you for your continued patience. Kind regards, Camila Kill Chief Technology Officer 1
DrCheese Posted October 22 Posted October 22 4 hours ago, Homer said: Ooooooh I bet you're on the money there. PHP 5.6 anyone? 😂 haha yes we said this, given the complete lack of development of the system in the past few years.
Tefters Posted October 22 Posted October 22 (edited) It's OK everybody, remember these companies are working in the best interest of schools. 🤣 Out of every product we buy into online this way the only one affected by using PHP and their bulk mailing supplier is so good that half the community got the emails WAY too late. I only just got the outage email at 21:30. Couldn't of simply redirected the classcharts DNS to a basic HTML holding page even if it was obvious why the outage had taken place. All this after a major wobble with the infrastructure the day before. Cloud computing is a beautiful thing until it isn't. Edited October 22 by Tefters
TechMonkey Posted October 23 Posted October 23 No inside knowledge of TES, purely speculation but when you think of the number of companies they have bought up in a pretty short space of time I very much doubt system, server or code base consolidation has happened yet fully, which means potetnitally each service is run on its own instance/cluster/raspberry pi. As said, this would mean each has to be updated individually, after checking what it might affect and then run tests as well as any supporting services they may have and general websites. I can imagine that is what is taking the time, and hopefully will highlight that they need to spend some of the money they are getting from numerous price hikes on the infrastructure.
Anyer Posted October 23 Posted October 23 Class Charts is back up, but Provision Map isn't which means none of the Pupil Passports on Class Charts are showing *sigh*
Homer Posted October 23 Posted October 23 11 hours ago, TechMonkey said: No inside knowledge of TES, purely speculation but when you think of the number of companies they have bought up in a pretty short space of time I very much doubt system, server or code base consolidation has happened yet fully, which means potetnitally each service is run on its own instance/cluster/raspberry pi. As said, this would mean each has to be updated individually, after checking what it might affect and then run tests as well as any supporting services they may have and general websites. I can imagine that is what is taking the time, and hopefully will highlight that they need to spend some of the money they are getting from numerous price hikes on the infrastructure. Raspberry Pi 😆
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now