[bromcom] Bromcom SSO Issue / Slowness

[Olliedawg]

Same here, printing out paper registers ready..

[Theblacksheep]

This is getting ridiculous and frankly embarrassing.

 

edit: it appears to be up again.

Edited January 23, 2024 by Theblacksheep

[timbo343]

Same here, printing out paper registers ready..

 

Yup, that's what our Admin team are doing too.

[kennysarmy]

So realistically what are the options if we move from Bromcom?

[SteveM555]

Not sure if the status page is updated manually.. "Degraded for 5 minutes"

[Olliedawg]

Not sure if the status page is updated manually.. "Degraded for 5 minutes"

 

I'd say its a manual thing, seems to be. I just had a notification saying that a Major Incident was resolved, however I'm still getting runtime errors when trying to log in

[CHiLL]

@Bromcom_Darren What is the escalation procedure for this seemingly never ending situation? It would be nice to know what the root cause of these issues are, such as not enough of certain resources, a certain aspect is a weak link, etc. Can we get at least get a commitment from the execs of your company to invest in the resources and/or expertise in order to get on top and apply some long-term solutions to stop this from being such a frequent occurrence?

 

I'm not just asking for myself, but on behalf of everyone else reading this, who is fed up with this not inexpensive solution not being able to meet expectations. It's clear that your team is not able to provide a consistent service or effectively fix the ongoing issues and I'm wondering if some form of refund for lack of service is something we can expect. I wouldn't be surprised if a bunch of schools or trusts migrated to Arbor when their contract with Bromcom expires.

Edited January 23, 2024 by CHiLL

[Rob_D]

Not sure if the status page is updated manually.. "Degraded for 5 minutes"

That "Degraded for..." counter just updates when you refresh the page.

 

Still seems pretty broken for us. Seems worse today that the normal Monday slowness.

[SteveM555]

That "Degraded for..." counter just updates when you refresh the page.

 

Still seems pretty broken for us. Seems worse today that the normal Monday slowness.

 

Ah I know, but at the time it was already down at least an hour and half, that's why I questioned if it was a manual update of the page.

[Cat_Jam148]

So realistically what are the options if we move from Bromcom?

Make sure your procurement is squeaky clean! Wouldn't want you to get sued by Bromcom...

[timbo343]

Broken here this morning too..

 

It feels like the saying against the horizon system, "no one else is having this problem".

 

And an email has landed:

 

"We have been made aware that some customers are experiencing issues with the MIS. This is being urgently investigated by the team."

 

The key word is some - more like everyone.

[snail5967]

We had staff trying to get in before 7.45 this morning - the case was only acknowledged just after 8am so obviously it's a monitoring problem at the Bromcom end otherwise they should have automatically picked it up as an issue. We could do with an update as to what has caused the issues and what is being done to prevent the reoccurance. We rely heavily on this cloud based solution so when it goes down, the entire school is in the dark, so to speak. When we had issues with the LGFL network a few weeks ago, we had reports from the CEO apologising and explaining the issue - will Bromcom do the same I wonder.......?

[Marci]

Refunds?

 

Most you'll get is an offer of SMS credits...

[Marci]

Ah I know, but at the time it was already down at least an hour and half, that's why I questioned if it was a manual update of the page.

 

Yes, it is.

[Olliedawg]

Has there been a downtime this long before? Unscheduled of course.

 

We're doing OK, have all registers printed off, but I can imagine its caused mayhem in some schools.

[timbo343]

We've had reports that the app is working fine but cloudmis.bromcom.com via a browser on a data connection doesn't work.

[Marci]

Has there been a downtime this long before? Unscheduled of course..

 

Yes, same point in time last January, on & off spanning a week or two.

[supportman]

Seems fixed now. 1hr 20mins downtime. ouch!

 

Always feels worse when its in the morning.

[Olliedawg]

Has there been an official update yet to say service restored? Last email I got was at 8:42 stating there was a major incident.

[michaeljbrewis97]

Bromcom marked the issues as resolved - so I think thats as official as it gets.

 

Has there been an official update yet to say service restored? Last email I got was at 8:42 stating there was a major incident.

[Marci]

Yes, at 9.35 for the OneUptime monitor...

 

If you logged your own Critical / SystemDown ticket, then it should have been marked resolved around the same time with a response along the lines of:

 

"We have had confirmation from our Cloud Team that the service is now up and stable. We will be providing customers with a root cause analysis once one is available. Thank you for your patience and apologies for the inconvenience caused."

 

...and Community members have been given pretty much the same statement. That's as much as we'll get for now.

 

They haven't issued a Customer Bulletin email (what you'll have had at 8.27ish).

Edited January 23, 2024 by Marci

[Marci]

Customer Bulletin Email has just landed at 9.59...

 

"Good morning,

 

Following on from our communication this morning, we have had confirmation from our Cloud Team that the service is now up and stable. We will be providing customers with a root cause analysis once one is available.

 

We sincerely apologise for any impact that you may have experienced with our service in the past few days. We understand the frustration and disruption these issues may have caused and we want to assure you that addressing these issues is our top priority.

 

We have been working on a significant programme of change within our organisation, with the overall objective being to improve the software & services that our customers rely upon, however, we acknowledge that as a result of these changes, we have inadvertently introduced issues.

 

We take full responsibility for this and are in the process of implementing additional governance and change management processes to ensure that these issues are not repeated.

 

Our fundamental priority at this time is to restore your confidence and trust in our software and services, and we sincerely hope that in the very short term, you will begin to see the positive benefits of these changes.System Updates will always be posted on the designated BromCommunity post as well as status.bromcomcloud.com

 

Best wishes,Bromcom Community Team"

Edited January 23, 2024 by Marci

[psydii]

Yes, same point in time last January, on & off spanning a week or two.

 

It'll be people running census reports.

[Marci]

*snigger*

[supportman]

Sounds actually like they are trying to improve performance and that is what has caused the issues.

 

Perhaps some kinds of backend upgrade that went wrong. Looking forward to reading the in depth analysis anyway. That's a good move at least.

 

As a network manager I can relate to that!