Arbor Down this morning

[Noxid_w]

I know I'm not the only school. Anyone else affected this morning?

 

Been nearly 2 hours. Anyone aware of what Arbor's SLA agreement is?

[Cat_Jam148]

We aren't affected by it here.

 

Have been keeping an eye on the incident report though

 

https://status.arbor-education.com/incidents/271299

[Primus]

We aren't affected by it here.

 

Have been keeping an eye on the incident report though

 

https://status.arbor-education.com/incidents/271299

 

Same here.

[sefton]

We've got one out of our eleven schools unavailable and I can't use the MAT site to log into any school site.

[Noxid_w]

Anyone gotten in back yet? Still nothing for us. I have found a document called Service Charter which lists what appears to be their SLA. I'll attach for everyone to peruse. They expect to not be down for more than 2 hours. We are at 3 hours and 50 minutes and counting.

 

https://support.arbor-education.com/hc/en-us/article_attachments/22153540241437

[LeMarchand]

Not affected any of my sites (thank goodness!).

[ovenchips]

No reports from any of my sites this morning, nothing from around the trust either!

[sefton]

The school site is live now and I can log in from the Trust site to any school.

[Noxid_w]

Been back now for 10 minutes. That's over 4 hours we've been down. Not a good experience. Will be interested to hear from Arbor what has happened.

[sefton]

Been back now for 10 minutes. That's over 4 hours we've been down. Not a good experience. Will be interested to hear from Arbor what has happened.

 

Database cluster constantly restarting.

[Ditto]

I'm not currently an Arbor user, or Bromcom, or any other MIS currently, but I take a keen interest in this stuff!

It is refreshing to have a look at Arbor's open status reporting, not even hidden behind a registered user account. They appeared, in the main, to be trying to meet their SLA's, but as so often is the case, they are targets that have no penalties for missing, but that's not uncommon.

If I recall the documentation correctly, today's incident will result in a Post Incident Report (PIR) to be public available within 5 days. I hope they keep to that commitment and we get more details that led to the cluster restart issues. I note in the last few days there have been several security updates following unauthorised access attempts - I wonder if there is some targeted attacking, but nothing to suggest it's linked to today's outage.

[mikes]

Damn, we are moving over from our local hosted SIMS (which being local we've never had a problem with) to Arbor - we've had problems with Schoolcloud going down during an online parents evening, and Classcharts going down for half a day once every 2 months or so. I hope we don't get this with Arbor as well. Our migration is LEA led so nothing to do with me so it won't fall on my head regardless.

 

I wonder if they are being DDOS'd or brute force attempts are happening? I hope they are transparent - one thing that worries me is what is Arbor security like? We will always have a teacher that writes their username and password down in huge letters on their laptop for every single student to see - with SIMS they'd need to have gotten a copy of the SIMS client but if it's web based, well we will see what the 2FA is like.

 

I've heard of stories of DN marketplaces being DDOS'd then being asked to pay ransom for it to stop - of course this never works (paying danegeld getting rid of the dane etc...) but a big company like Arbor should be able to handle it I'd think. Be interesting to read the PIR as we get ready to migrate over. Do they keep a list of past PIR's ? (edit they do at least that is something. I like how they stopped all outside UK access for a period due to an incident - better safe than sorry)

Edited February 5, 2025 by mikes

[AlistairB1983]

Damn, we are moving over from our local hosted SIMS (which being local we've never had a problem with) to Arbor - we've had problems with Schoolcloud going down during an online parents evening, and Classcharts going down for half a day once every 2 months or so. I hope we don't get this with Arbor as well.

We've been with them just under a year and we've never had a complete outage issue. We've had a couple of instances where some members of staff couldn't access it for about an hour or so, but nothing more than that.

 

(Obviously we weren't one of the affected schools yesterday, but I appreciate others were and, judging by the repeated email updates, they were affected for quite some time).

 

Generally they seem to be doing a much better job in terms of uptime compared to when I used them in a previous school a few years back. At that point it was very much a frequent issue, but recently not so much. Finger's crossed you experience the same once you complete your move.

[psydii]

As long as a powerful head/deputy owns the project, you'll be fine, even if you have a few bumpy days a year. If it was sold as "well IT said we had to move to this new thing", then riding out any bad days might be tough - slt and teachers can be an unforgiving audience.

 

Always always have a backup plan for registers ready to go at a moments notice. Always have some (backup) independent system for keeping track of students on the safeguarding register. Always have a independent backup system for staff and student contacts. Pretty much everything else you can survive (i.e. keep the school open) without.

 

[EDIT] Exam entries - you need a backup system for these too. JQC get very grumpy if you don't.

Edited February 5, 2025 by psydii

[sefton]

Damn, we are moving over from our local hosted SIMS (which being local we've never had a problem with) to Arbor - we've had problems with Schoolcloud going down during an online parents evening, and Classcharts going down for half a day once every 2 months or so. I hope we don't get this with Arbor as well. Our migration is LEA led so nothing to do with me so it won't fall on my head regardless.

 

I wonder if they are being DDOS'd or brute force attempts are happening? I hope they are transparent - one thing that worries me is what is Arbor security like? We will always have a teacher that writes their username and password down in huge letters on their laptop for every single student to see - with SIMS they'd need to have gotten a copy of the SIMS client but if it's web based, well we will see what the 2FA is like.

 

I've heard of stories of DN marketplaces being DDOS'd then being asked to pay ransom for it to stop - of course this never works (paying danegeld getting rid of the dane etc...) but a big company like Arbor should be able to handle it I'd think. Be interesting to read the PIR as we get ready to migrate over. Do they keep a list of past PIR's ? (edit they do at least that is something. I like how they stopped all outside UK access for a period due to an incident - better safe than sorry)

 

I've got 11 schools and have been with them for 3 years now and this is the first serious outage I have had, just the one junior school though, so manageable.