Diamond Notes

Just another WordPress weblog

Scaling MySQL - - Up or Out? Panel @ UC

I would recommend that you download the video of this!! Sheeri posted it here.

The numbers in parentheses are Alexa rankings.

Moderator - Kaj Arno

(1317) Monty Taylor - MySQL

(905) Matt Ingenthron - Sun

(39) John Allspaw - Flickr

(13) Frank mash - Fotolog

(9) Domas Mituzas - Wikipedia

(6) Jeff Rothschild - Facebook

(2) Paul Tuckfield - YouTube

Question One: Number of MySQL servers

MySQL one master/three slaves

Sun four servers

Flickr 166

Fotolog 37

Wikipedia

Facebook 1,800 (900m/900s)

YouTube


Question Two: Number of MySQL DBAs

MySQL 1/10th

Sun 1.5

Flickr 0 (normally 1)

Fotolog 1

Wikipedia Technical Team

Facebook 2

YouTube 3

Question Three: Number of Web Servers

MySQL 2

Sun 160

Flickr 244

Fotolog 70

Wikipedia

Facebook 10,000

YouTube

Question Four: Number of Memcached servers

MySQL 2

Sun 8

Flickr 14

Fotolog 40

Wikipedia 79

Facebook 805

YouTube

Question Five: Version of MySQL

MySQL 5.23-2rc

Sun 5.0.21

Flickr 5.0.51

Fotolog 4.11

Wikipedia 4.4

Facebook 5.0.44

YouTube 5.0.24

Question Six: Operating System on Server

MySQL Fedora

Sun OpenSolaris

Flickr Linux

Fotolog Solaris 10

Wikipedia Fedora/Ubuntu

Facebook Fedora/RHEL

YouTube SuSE 9

Question Seven: What happens if a server fails?

Flickr - Federated setup for failover. Can loose any one side of the shard.

Wikipedia - if a master fails they replace with slave

Facebook - archive binlogs, promote slave

Fotolog - mount snapshots?

Youtube - SAN; shards with a master and multiple slaves so they promote slaves

Question Eight: What is Their Crucial Scaling Technology

Facebook doesn’t use SAN - they do use RAID 10 with 2.5″ drives

Fotolog — UltraSparc T1 — excellent master UltraSparc T2 — excellent slave — uses SAN

This was interesting to me. Frank (Fotolog) said they use a SAN to keep things manageable (only two dbas with the second one just hired). Facebook says they don’t use SAN because they didn’t want to limit themselves.
Next they got off on discussion about power. This varied quite a bit with YouTube pretty much dismissing power concerns. Of course Frank from Fotolog then pointed out that when they (Fotolog) want to expand in a datacenter — the datacenter has to get Google’s approval…hmmm..no wonder Google isn’t worried about it. Fotolog and Facebook were very much in favor of power savings. I think there is more than just saving a little power, you get cooling and space (if smaller of course) savings.

29 Comments so far

  1. […] has a nice summary of everything that went on together with the numbers here. Posted by Artem Russakovskii in […]

  2. Artem Russakovskii April 16th, 2008 4:10 pm

    Thanks for publishing this, I was looking for the notes. These numbers are fun.

  3. MySQL Expo - Day 2 at Jeremy’s Blog April 16th, 2008 4:20 pm

    […] (Google is still quite secretive about many numbers-related items it seems). Good coverage of the numbers presented (Colin was plogging, so that document should be released soon and the video should be available on […]

  4. Frank April 16th, 2008 5:10 pm

    Hi,

    Thanks for writing this. A few corrections:

    1. We have 37 database servers. The number posted is the number of instances.

    2. In the following sentence, Facebook should be replaced with Fotolog, as it was me who made the point :)

    “Of course Facebook then pointed out that when they (Facebook) want to expand in a datacenter — the datacenter has to get Google’s approval…hmmm..no wonder Google isn’t worried about it.”

    3. Our second DBA has yet to start :)

    Thanks

  5. Frank April 16th, 2008 5:11 pm

    Ughhh, typo in my last comment:

    We have 37 database servers. The number posted is the number of databases.

    thanks!

  6. admin April 16th, 2008 5:53 pm

    Thanks for the updates Frank. Sorry about the mistake. I updated the numbers.

    Keith

  7. frogcake.net » Conferences April 17th, 2008 4:47 am

    […] I can live vicariously through all the people blogging the event. Artem Russakovskii’s notes on the Scaling MySQL - Up or Out? keynote have some interesting numbers: Question One: Number of […]

  8. Bob Jones April 18th, 2008 10:20 am

    They should be running IIS/Windows/MSSQL. Scales better.

  9. Peteris Krumins April 18th, 2008 1:11 pm

    I’m sorry, but where is the video you are talking about?

  10. links for 2008-04-18 April 18th, 2008 1:32 pm

    […] Diamond Notes » Scaling MySQL - - Up or Out? Panel @ UC (tags: configurations mysql server) […]

  11. Cyndy Aleo-Carreira April 18th, 2008 4:18 pm

    So Facebook is built on Rails? I can’t even fathom what their burn rate must be what with the employees, the perks, and the unholy number of web servers.

  12. Ken Robertson April 18th, 2008 5:59 pm

    Cyndy: Facebook is built on PHP: http://blog.facebook.com/blog.php?post=2356432130

  13. links for 2008-04-19 | hxf148 April 19th, 2008 3:34 am

    […] Diamond Notes » Scaling MySQL - - Up or Out? Panel @ UC […]

  14. domas mituzas April 19th, 2008 4:23 am

    database server count: twenty :)

    thats for wikipedia

  15. Artem Russakovskii April 19th, 2008 8:03 am

    Who mentioned rails and where? Facebook is built on php.

  16. Cyndy Aleo-Carreira April 19th, 2008 12:23 pm

    Artem, I was joking at the Rails theory of throwing more servers at a problem. Facebook has an insane number of servers and MySQL instances based on their approximate pageviews.

  17. admin April 19th, 2008 7:16 pm

    Updated with link to video.

  18. […] Среди новостей с прошедшей на прошлой неделе MySQL Conference в Санта-Кларе - кто сколько баз данных MySQL пользует в своих проектах: […]

  19. […] Number of Servers at Wikipedia, Facebook, FLickr and YouTube Number of MySQL servers Flickr 166 Facebook 1,800 (900m/900s) (tags: servers scalability mysql) […]

  20. Perspectives April 22nd, 2008 8:36 am

    1,800 MySQL Servers with Two DBAs…

  21. […] Miller points out some notes from a recent MySQL user conference during which Facebook VP of Technology Jeff Rothschild gave […]

  22. […] servers, according to Data Center Knowledge, citing comments made by Facebook VP of technology, Jeff Rothschild, at a recent MySQL user conference. (See video of the panel.) Of the 10,000 servers, 1,800 are from […]

  23. […] a recent MySQL conference, titled “Scaling MySQL - Up or Out”, Facebook revealed that they are currently operating 10,000 web servers and 1,800 database servers. Another newsworthy […]

  24. […] servers, according to Data Center Knowledge, citing comments made by Facebook VP of technology, Jeff Rothschild, at a recent MySQL user conference. (See video of the panel.) Of the 10,000 servers, 1,800 are from […]

  25. […] Diamond Notes » Scaling MySQL - - Up or Out? Panel @ UC […]

  26. baanmo.com May 5th, 2008 11:22 am

    Strategy: Break Up the Memcache Dog Pile…

  27. Aaron Johnson – Links: 5-6-2008 May 7th, 2008 1:53 am

    […] Diamond Notes » Scaling MySQL - - Up or Out? Panel @ UC Wow. Facebook = 1,800 MySQL servers, 10,000 http servers, 800 memcached servers. (categories: facebook scaling memcached mysql performance architecture ) […]

  28. Niels Olson May 23rd, 2008 5:23 am
  29. Facebook Videos | Long Blog June 26th, 2008 7:15 am

    […] files. 540 terabytes. 475 000 photos served per second. 100 million photos uploaded every week. 10 000 servers. Those are big ass numbers with big ass […]

Leave a reply