Calgarypuck Forums - The Unofficial Calgary Flames Fan Community
Old 10-04-2021, 02:20 PM   #21
Firebot
First Line Centre
 
Join Date: Jul 2011
Exp:
Default

https://twitter.com/user/status/1445078208190291973
Firebot is offline   Reply With Quote
Old 10-04-2021, 02:22 PM   #22
Cecil Terwilliger
That Crazy Guy at the Bus Stop
 
Cecil Terwilliger's Avatar
 
Join Date: Jun 2010
Location: Springfield Penitentiary
Exp:
Default

Quote:
Originally Posted by burn_this_city View Post
IT version of an inside job to bring down the company?
Live shot inside the FB headquarters:

Cecil Terwilliger is offline   Reply With Quote
The Following 9 Users Say Thank You to Cecil Terwilliger For This Useful Post:
Old 10-04-2021, 02:23 PM   #23
Minnie
Franchise Player
 
Minnie's Avatar
 
Join Date: Dec 2012
Location: On your last nerve...:D
Exp:
Default

Quote:
Originally Posted by Skaloper View Post
Looks like everyone went to Twitter...
They had fun with it. A lot of the big brands joined in. Made me laugh a few times.

https://twitter.com/user/status/1445078208190291973
__________________
No.

Last edited by Minnie; 10-04-2021 at 02:27 PM.
Minnie is offline   Reply With Quote
Old 10-04-2021, 02:25 PM   #24
Hack&Lube
Atomic Nerd
 
Join Date: Jul 2004
Location: Calgary
Exp:
Default

ELI summary of what it sounds like to me:

1. There are Facebook datacenters all around the world with routers that tell internet on the traffic how to get to the facebook servers

2. Someone or some process automatically updated all these routers with the wrong config and now there is no way to connect to them

3. All these datacenters have security systems that are also tied to the internet and this broke all the authentication for keycard access so they can't actually get in to physically fix the problem

If Facebook were a fleet of 10,000 cars, this would be like all the drivers locking their keys in their car all at the same time and not being able to get back in.
Hack&Lube is offline   Reply With Quote
The Following 2 Users Say Thank You to Hack&Lube For This Useful Post:
Old 10-04-2021, 02:25 PM   #25
Nyah
First Line Centre
 
Nyah's Avatar
 
Join Date: May 2012
Location: The Kilt & Caber
Exp:
Default

T'would be glorious, but isn't it super easy and quick to fix routing issues?

https://twitter.com/user/status/1445104389014835213
Nyah is offline   Reply With Quote
Old 10-04-2021, 02:26 PM   #26
Hack&Lube
Atomic Nerd
 
Join Date: Jul 2004
Location: Calgary
Exp:
Default

Quote:
Originally Posted by Nyah View Post
Facebook would have backups of the routing tables if they could only get to the physical hardware to restore them. If they didn't then WHOOHOO!
Hack&Lube is offline   Reply With Quote
Old 10-04-2021, 02:27 PM   #27
Wastedyouth
Truculent!
 
Join Date: Aug 2013
Exp:
Default

Quote:
Originally Posted by Hack&Lube View Post
It sounds like BGP peering with the Facebook peering routers is down and this happened due to a configuration change via some kind of automation. This would break all DNS and routing. This may have taken down the peering routers completely and since they can no-longer be accessed remotely, they need to be accessed physically inside the Facebook datacenters and this is causing further issues since datacenter access is severely restricted and staffing is all remote due to Covid-19.

Basically if their BGP routers withdrew their DNS records or are no-longer advertising FB as a valid route, nobody can get to their DNS records to route traffic to facebook. This seems to have propagated everywhere instantly. What's worse is their internal systems were also using this so nobody can access their buildings or facilities to actually fix the problem directly.

I found this cached article where Facebook engineering is lauding their peering automation. This sounds like the candidate for what broke: https://webcache.googleusercontent.c...&ct=clnk&gl=ca

Some tech friends were like "haha!" until they were like "nooo my stonks!"
You are an atomic nerd.
__________________
Quote:
Originally Posted by Poe969 View Post
It's the Law of E=NG. If there was an Edmonton on Mars, it would stink like Uranus.
Wastedyouth is offline   Reply With Quote
Old 10-04-2021, 02:32 PM   #28
Fuzz
Franchise Player
 
Fuzz's Avatar
 
Join Date: Mar 2015
Exp:
Default

It's probably not quite as easy as just putting a new configuration in. With traffic demand as big as Facebook, if you, say, start with one BGP router, that route gets hammered, and will probably take down nodes as all Facebook traffic hits one target. So you would have to be careful about restoring it. Hopefully they have a disaster recovery plan for this scenario, or it might cause problems for days.
__________________
Air Canada - We're not happy until you're not happy.
Telus - Almost as bad as Winnipeg.
Calgary Roads Dept - Ya, we'll get to that.
Fuzz is online now   Reply With Quote
Old 10-04-2021, 02:38 PM   #29
Nyah
First Line Centre
 
Nyah's Avatar
 
Join Date: May 2012
Location: The Kilt & Caber
Exp:
Default

Quick! Everyone get your idiot relatives who get all their information from FB vaccinated before it's back up again!
Nyah is offline   Reply With Quote
The Following 6 Users Say Thank You to Nyah For This Useful Post:
Old 10-04-2021, 02:39 PM   #30
calumniate
Franchise Player
 
calumniate's Avatar
 
Join Date: Feb 2007
Location: A small painted room
Exp:
Default

https://www.newyorker.com/humor/boro...we-own-at-risk

Paywall - but good stuff from the new yorker
__________________
You hardly know a day goes by in the cardboard cut-out sundown
calumniate is offline   Reply With Quote
Old 10-04-2021, 02:40 PM   #31
Ozy_Flame

Posted the 6 millionth post!
 
Ozy_Flame's Avatar
 
Join Date: Feb 2002
Exp:
Default

https://twitter.com/user/status/1445123786743984148
Ozy_Flame is offline   Reply With Quote
The Following 2 Users Say Thank You to Ozy_Flame For This Useful Post:
Old 10-04-2021, 02:42 PM   #32
Senator Clay Davis
Franchise Player
 
Senator Clay Davis's Avatar
 
Join Date: Apr 2012
Location: Maryland State House, Annapolis
Exp:
Default

Zuckerberg kinda looks like a real life Lionel Hutz lol.
__________________
"Think I'm gonna be the scapegoat for the whole damn machine? Sheeee......."
Senator Clay Davis is offline   Reply With Quote
Old 10-04-2021, 03:13 PM   #33
Flabbibulin
Franchise Player
 
Flabbibulin's Avatar
 
Join Date: Nov 2009
Exp:
Default

Quote:
Originally Posted by Hack&Lube View Post
If Facebook were a fleet of 10,000 cars, this would be like all the drivers locking their keys in their car all at the same time and not being able to get back in.
Or you could just say it is like facebook was:

Flabbibulin is offline   Reply With Quote
The Following 2 Users Say Thank You to Flabbibulin For This Useful Post:
Old 10-04-2021, 03:15 PM   #34
TorqueDog
Franchise Player
 
TorqueDog's Avatar
 
Join Date: Jul 2010
Location: Calgary - Centre West
Exp:
Default

Part of me is happy about this, well deserved for how destructive Facebook has been to society as a whole.

Another part of me is thinking "ah ####, there goes anything I used OAuth through Facebook for".
__________________
If the Flames are playing, GO FLAMES GO.
If the Jets are playing, GO JE-... nm, ғ**ĸ the Jets.
GO FLAMES GO.
TorqueDog is offline   Reply With Quote
Old 10-04-2021, 03:20 PM   #35
Calgary14
Powerplay Quarterback
 
Join Date: Jan 2014
Exp:
Default

I think life is actually better without Facebook
Calgary14 is offline   Reply With Quote
The Following User Says Thank You to Calgary14 For This Useful Post:
Old 10-04-2021, 03:21 PM   #36
Flabbibulin
Franchise Player
 
Flabbibulin's Avatar
 
Join Date: Nov 2009
Exp:
Default

Quote:
Originally Posted by Calgary14 View Post
I think life is actually better without Facebook
How the hell am I gonna wish my mother a happy birthday?
Flabbibulin is offline   Reply With Quote
Old 10-04-2021, 03:22 PM   #37
dino7c
Franchise Player
 
Join Date: Jul 2011
Exp:
Default

Where am I going to get my medical advice though
__________________
I WAS WRONG ABOUT KYLINGTON
dino7c is offline   Reply With Quote
The Following User Says Thank You to dino7c For This Useful Post:
Old 10-04-2021, 03:22 PM   #38
TheIronMaiden
Franchise Player
 
TheIronMaiden's Avatar
 
Join Date: May 2016
Location: ATCO Field, Section 201
Exp:
Default

Quote:
Originally Posted by Calgary14 View Post
I think life is actually better without Facebook
I haven't had facebook for 3-4 years and I agree completely.
TheIronMaiden is offline   Reply With Quote
The Following 2 Users Say Thank You to TheIronMaiden For This Useful Post:
Old 10-04-2021, 03:24 PM   #39
Hack&Lube
Atomic Nerd
 
Join Date: Jul 2004
Location: Calgary
Exp:
Default

I also stopped using Facebook for the last 5-6 years but I heavily use messenger to stay in touch with people.
Hack&Lube is offline   Reply With Quote
Old 10-04-2021, 03:46 PM   #40
Fuzz
Franchise Player
 
Fuzz's Avatar
 
Join Date: Mar 2015
Exp:
Default

Aww crap, it looks like ti might be coming back to life. Kill it! Kill it with fire!
__________________
Air Canada - We're not happy until you're not happy.
Telus - Almost as bad as Winnipeg.
Calgary Roads Dept - Ya, we'll get to that.
Fuzz is online now   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -6. The time now is 06:00 PM.

Calgary Flames
2021-22




Powered by vBulletin® Version 3.8.4
Copyright ©2000 - 2021, Jelsoft Enterprises Ltd.
Copyright Calgarypuck 2021