27 points saikatsg 2 hours ago 9 comments

christhecaribou 2 hours ago | parent

This is one for the history books. The status updates seem like AI nonsense, it doesn’t seem like they’ve even root-caused what’s driving this.

xvxvx 2 hours ago | parent

Gmail down for me also. China must be mad about the Nvidia dude.

proc0 2 hours ago | parent

Looking forward to the postmortem on this one and how much AI contributed to the problem and troubleshooting.

I've been saying AI hasn't been tested properly. All the showcases I see aren't really pushing it far enough and they're always things you could in theory do by searching answers online. The real value is in all the problems that cannot be solved by online search.

Eji1700 1 hour ago | parent

I’m extremely curious what actual veterans could even come up with to explain the situation as it currently appears.

I’ve seen the chats about cascading failures and it’s impossible to not expect AI is involved somehow (likely brain drain and what not), but what are the options?

It sure looks like “fired everyone who knew what they were doing” but I feel like at this point you’re breaking open the check book and calling them, and even if they did that I’m assuming this shouldn’t be a “normal” event?

borzi 1 hour ago | parent

Hear that - that is the sound of every Engineer at Microsoft right now typing "We have a global incident - fix it" furiously into their chatbots!

moomoo11 1 hour ago | parent

hey Claude fix my production issue in production make no mistakes

investigating..

goofing..

finagling..

hey claude what the hell you deleted all my configs permanently!!

metastasizing.. You’re absolutely right. I went above and beyond what you had asked and started completely fresh. Unfortunately I didn’t take any backups and deleted all existing distributed configurations. Oopsie!

karlkloss 1 hour ago | parent

You don't say.

dwedge 30 minutes ago | parent

I know this is only tenuously related, but after a week of them making changes (starting with DKIM) and blocking third party mailservers (and this week I got my first "your IP is blocked" but when you use their request to remove they say it isn't), I hope it goes down long enough to make more people migrate away

cedws 19 minutes ago | parent

Get Copilot to fix it.