I think google accounts has just gone down

[jthompson]

My Gmail mail delivery check is now happy again, although some things are still iffy. I'm not yet able to access delegated mailboxes, for instance, and Drive File Stream keeps failing authentication.

 

[TMBS]

I’m not sure I would trust that. It shows no outages where I am...

It was more accurate than the Google Workspace Dashboard which was showing full green at the time...

[Koldov]

@Garacesh

 

I think it might be the Solar Winds thing...

 

https://www.crn.com/news/security/fireeye-us-federal-agencies-hacked-through-solarwinds-report

 

Also picked up here...

 

http://www.edugeek.net/forums/security/219251-solarwinds-fireeye.html

[Garacesh]

My Gmail mail delivery check is now happy again, although some things are still iffy. I'm not yet able to access delegated mailboxes, for instance, and Drive File Stream keeps failing authentication.

 

[ATTACH=CONFIG]60178[/ATTACH]

 

My File Stream has signed me back in. GMail seems to be up again for me, too.

[Rob_D]

Google's working again for us.

[paulkerton]

It was more accurate than the Google Workspace Dashboard which was showing full green at the time...

Status Dashboard is purely manual. It doesn't run automatically.

Plus, I'm guessing no ones login was working to be able to get in to update it?

[jthompson]

Everything seems okay apart from Gmail for our primary admin account, which is still not available. Google status dashboard's final update on it was over half an hour ago. Anyone else currently still having any issues with Gmail?

 

Edit: As soon as I posted that, it then started working again.

Edited December 14, 2020 by jthompson

[SpaceInvader83]

My Gmail mail delivery check is now happy again, although some things are still iffy. I'm not yet able to access delegated mailboxes, for instance, and Drive File Stream keeps failing authentication.

 

[ATTACH=CONFIG]60178[/ATTACH]

 

 

Is this using Zabbix?

[paulkerton]

I see The Daily Express are calling it the "9/11 of Hacks"

Because we all know their stirling reputation for calm and measured responses...

[MrEprise]

Everything seems okay apart from Gmail for our primary admin account, which is still not available. Google status dashboard's final update on it was over half an hour ago. Anyone else currently still having any issues with Gmail?

 

Edit: As soon as I posted that, it then started working again.

 

Their 'Status Dashboard' shows an orange circle under GMail so maybe they were still having issues with that.

[psydii]

I see The Daily Express are calling it the "9/11 of Hacks"

Because we all know their stirling reputation for calm and measured responses...

 

It could be. The hackers may have had persistence since mid 2019.

 

FireEye has an arsenal of tools that were stolen, and a huge amount of knowledge of some of the biggest private and governmental IT systems.

 

SolarWinds Orion is in many sensitive networks.

 

Any network with Orion will likely have had DA, and SSO authentication tokens stolen.

 

I do wonder if the MS and Google outages are in fact due to rotating certificates/tokens, done today because un-nameable orgs had enough time over the weekend to re-secure their networks and were ready to reset accounts and passwords again.

 

(If your DA gets compromised (golden ticket) you have to change your KTGT password *twice* in order to prevent the hackers regaining persistence, I have very little idea how you recover if global administrator tokens are got.)

 

https://cyber.dhs.gov/ed/21-01/

[localzuk]

Looks like it was some form of quota issue.

 

[jthompson]

Is this using Zabbix?

 

No, it's Check_MK (https://checkmk.com/cms_check_check_mail_loop.html).

[psydii]

Looks like it was some form of quota issue.

 

 

 

Hey. Never let reasonably likely explanations get in the way of a good global conspiracy.