Classcharts abysmal performance today

[WannabrewUK]

I chased them this morning and got a response pretty quickly.

 

"We have deployed a custom field fix for all customers and, although we are still working through individual scenarios, the majority of custom fields should now be available. We will of course continue to monitor the situation and provide any further updates where necessary."

 

Some other parts appear to still be broken.

[PotNoodleTech]

Classcharts down again - same nginx error as last time

[Anyer]

Classcharts down again - same nginx error as last time

Same here, but it has just come back up for us. Looks like we might in for a sporadic up and down kind of day.

 

Also not had any kind of explanation from them for last week's downtime.

[PotNoodleTech]

Yep back up here too, Theres nothing more annoying than up and down like a yo yo.

[Matt007a]

We're now missing loads of kids photos all of a sudden since it came back up.

[3s-gtech]

Quote

We are aware of some customers experiencing Internal Errors whilst using Class Charts. Our engineers are working on this issue as a priority. Please accept our apologies for the inconvenience caused. Where possible, we would suggest you use your MIS for recording attendance whilst we resolve the issue.

You pay for our product, but we suggest that you don't use our product for the designed purpose.

 

Righto chief!

[newpersn]

Edukey / Provmaps has been up and down all morning! 

[psydii]

I've just onboarded the attendance team to SIMS NG, ready to pivot for tomorrow.

[psydii]

Before anyone mentions it,  the Goggle Cloud Services thing should be irrelevant, because they don't declare that as part of the platform. (though it is used for some internal analytics productivity and advertising purposes).

 

 

[psydii]

Anyone heard if Classcharts are expecting to have stability tomorrow? or does the "use your mis" advice stand, and I need to on board everyone to SIMS NG for tomorrow?

[3s-gtech]

What do you mean, heard from Class Charts? They do not communicate, especially with the plebs. They will change the status when they're good and ready.

 

Get them using SIMS instead of CC, then hit TES at renewal.

[NetworkDirector]

3 hours ago, newpersn said:

Edukey / Provmaps has been up and down all morning! 

Mobile apps and SSO run on google cloud with class charts, could explain the slowness.

[psydii]

1 minute ago, NetworkDirector said:

Mobile apps and SSO run on google cloud with class charts, could explain the slowness.

Hmm. Is that in the list of 3rd parties etc in the the data sharing agreement/privacy notice?

[NetworkDirector]

16 minutes ago, psydii said:

Hmm. Is that in the list of 3rd parties etc in the the data sharing agreement/privacy notice?

No but Google sign in is using Google Cloud and as I am aware they use Google Analytics, and also remember ClassCharts will rely on other services where it may use Google Cloud. 

[psydii]

So after a good start today, it’s starting to wobble again. I’d log it but as mentioned above they don’t talk to the plebs.

 

 

[3s-gtech]

We've logged a job as it hasn't synced anything for days. Wonde broke, they've hopefully fixed that but my god - I hate trying to negotiate with staff about why something I didn't make, purchase or control doesn't work. Hell I only had admin a few weeks ago. Nor do I have access to the attendance data to know that it's not working.

Edited yesterday at 12:03 by 3s-gtech

[psydii]

Writeback was broken yesterday, but seems to be working today.  We did deploy CDPsu3, but nothing in that seems like it would have had any impact at all on sync issues.

 

I quite enjoy it being "not my problem"  It helps open SLT's eyes to reality with regard IT service/operations challenges. 

That said CDPsu4 is being developed "as a matter of utmost urgency... [and will be released] ...tomorrow"! Hopefully nothing messed up in SIMS  - because that would be my problem (for now).

[NetworkDirector]

ClassCharts is currently suffering issues with Writeback to SIMS and other MIS. Also be aware that ClassCharts and many other web applications may be suffering downtime or updates over the next few days due to a PHP remote execution vulnerability, just to make people aware so they aren't frantically wondering why loads of apps are down or websites. So check your Intranets are not using PHP, or update it at least to the latest version.

Edited yesterday at 15:26 by NetworkDirector
Spelling error

[m_m]

Still down this morning.

[rpenhale]

All due to this:-

 

PHP devs publish critical bug fix as site owners warned: Pull 'em offline if you're unpatched

 

Dear Integration,
 
We want to let you know about a temporary service disruption affecting much of the internet and online services worldwide.
 
This is due to a global issue with widely used internet technology that sits behind thousands of online services. There is no evidence that Tes has been directly affected, however we’ve taken some services temporarily offline as a precaution while we monitor this global situation.
 
What this means for you:
 
Some Tes pages, resources or services may not load for now. We’re pausing these services as a precaution, and we have no evidence that anything has been impacted. We expect everything to be back online as normal and as soon as possible.
 
There’s nothing you need to do at your end. We’ll take care of everything and let you know as soon as services are restored.
 
Our team is monitoring the situation and will restore access as quickly as possible.
 
Thank you for your patience.
 
Kind regards,
Camila Kill
Chief Technology Officer, Tes

[gomalley]

17 minutes ago, rpenhale said:

All due to this:-

 

PHP devs publish critical bug fix as site owners warned: Pull 'em offline if you're unpatched

 

Dear Integration,
 
We want to let you know about a temporary service disruption affecting much of the internet and online services worldwide.
 
This is due to a global issue with widely used internet technology that sits behind thousands of online services. There is no evidence that Tes has been directly affected, however we’ve taken some services temporarily offline as a precaution while we monitor this global situation.
 
What this means for you:
 
Some Tes pages, resources or services may not load for now. We’re pausing these services as a precaution, and we have no evidence that anything has been impacted. We expect everything to be back online as normal and as soon as possible.
 
There’s nothing you need to do at your end. We’ll take care of everything and let you know as soon as services are restored.
 
Our team is monitoring the situation and will restore access as quickly as possible.
 
Thank you for your patience.
 
Kind regards,
Camila Kill
Chief Technology Officer, Tes

How did you get this - can't get them to notify us of a thing!

Also - what on earth does "We expect everything to be back online as normal and as soon as possible." even mean?!?

[rpenhale]

Just now, gomalley said:

How did you get this - can't get them to notify us of a thing!

Also - what on earth does "We expect everything to be back online as normal and as soon as possible." even mean?!?

I guess historically we must have signed up for a notification email, came from this email address [email protected]

[gomalley]

Lucky you! I've raised their lack of comms so many times, and yet they obviously have a list they still email, just don't let you join! Can't believe I'm saying it but can't wait for our attendance to move to SIMS Next Gen at this point!

[gomalley]

TES public website is back - guess they feel that is more important than our services!

[Anyer]

3 hours ago, rpenhale said:

I guess historically we must have signed up for a notification email, came from this email address [email protected]

I never get anything as the main admin for Class Charts, yet now I know the email I did a message trace and everyone in Senior Leadership got the email bar me! When it's all up and running I'm going to get them to add me to this notification email list.