I would recommend that you download the video of this!! Sheeri posted it here.
The numbers in parentheses are Alexa rankings.
Moderator – Kaj Arno
(1317) Monty Taylor – MySQL
(905) Matt Ingenthron – Sun
(39) John Allspaw – Flickr
(13) Frank mash – Fotolog
(9) Domas Mituzas – Wikipedia
(6) Jeff Rothschild – Facebook
(2) Paul Tuckfield – YouTube
Question One: Number of MySQL servers
MySQL one master/three slaves
Sun four servers
Flickr 166
Fotolog 37
Wikipedia
Facebook 1,800 (900m/900s)
YouTube
Question Two: Number of MySQL DBAs
MySQL 1/10th
Sun 1.5
Flickr 0 (normally 1)
Fotolog 1
Wikipedia Technical Team
Facebook 2
YouTube 3
Question Three: Number of Web Servers
MySQL 2
Sun 160
Flickr 244
Fotolog 70
Wikipedia
Facebook 10,000
YouTube
Question Four: Number of Memcached servers
MySQL 2
Sun 8
Flickr 14
Fotolog 40
Wikipedia 79
Facebook 805
YouTube
Question Five: Version of MySQL
MySQL 5.23-2rc
Sun 5.0.21
Flickr 5.0.51
Fotolog 4.11
Wikipedia 4.4
Facebook 5.0.44
YouTube 5.0.24
Question Six: Operating System on Server
MySQL Fedora
Sun OpenSolaris
Flickr Linux
Fotolog Solaris 10
Wikipedia Fedora/Ubuntu
Facebook Fedora/RHEL
YouTube SuSE 9
Question Seven: What happens if a server fails?
Flickr – Federated setup for failover. Can loose any one side of the shard.
Wikipedia – if a master fails they replace with slave
Facebook – archive binlogs, promote slave
Fotolog – mount snapshots?
Youtube – SAN; shards with a master and multiple slaves so they promote slaves
Question Eight: What is Their Crucial Scaling Technology
Facebook doesn’t use SAN – they do use RAID 10 with 2.5″ drives
Fotolog — UltraSparc T1 — excellent master UltraSparc T2 — excellent slave — uses SAN
This was interesting to me. Frank (Fotolog) said they use a SAN to keep things manageable (only two dbas with the second one just hired). Facebook says they don’t use SAN because they didn’t want to limit themselves.
Next they got off on discussion about power. This varied quite a bit with YouTube pretty much dismissing power concerns. Of course Frank from Fotolog then pointed out that when they (Fotolog) want to expand in a datacenter — the datacenter has to get Google’s approval…hmmm..no wonder Google isn’t worried about it. Fotolog and Facebook were very much in favor of power savings. I think there is more than just saving a little power, you get cooling and space (if smaller of course) savings.
Thanks for publishing this, I was looking for the notes. These numbers are fun.
Hi,
Thanks for writing this. A few corrections:
1. We have 37 database servers. The number posted is the number of instances.
2. In the following sentence, Facebook should be replaced with Fotolog, as it was me who made the point
“Of course Facebook then pointed out that when they (Facebook) want to expand in a datacenter — the datacenter has to get Google’s approval…hmmm..no wonder Google isn’t worried about it.”
3. Our second DBA has yet to start
Thanks
Ughhh, typo in my last comment:
We have 37 database servers. The number posted is the number of databases.
thanks!
Thanks for the updates Frank. Sorry about the mistake. I updated the numbers.
Keith
They should be running IIS/Windows/MSSQL. Scales better.
I’m sorry, but where is the video you are talking about?
So Facebook is built on Rails? I can’t even fathom what their burn rate must be what with the employees, the perks, and the unholy number of web servers.
Cyndy: Facebook is built on PHP: http://blog.facebook.com/blog.php?post=2356432130
database server count: twenty
thats for wikipedia
Who mentioned rails and where? Facebook is built on php.
Artem, I was joking at the Rails theory of throwing more servers at a problem. Facebook has an insane number of servers and MySQL instances based on their approximate pageviews.
Updated with link to video.
better link to the video:
http://www.technocation.org/content/panel-video%3A-scaling-mysql-or-out%3F
10,000 database servers for Facebook!? At a rough guess they would get 30,000 DB transactions per second… Having 10,000 servers seems a bit over the top don’t you think?
Flickr || 166 || 244
Maybe I missed something, but does this mean that Flickr requires only 244 (or 410 = 244 + 166?) servers to work?
The _whole_ Flickr with its millions of photos?!