Stuff The Internet Says On Scalability For February 11, 2011

Submitted for your reading pleasure...

• A good night's sleep is why Facebook CTO Bret Taylor says Friendfeed should have gone cloud, let others take the midnight watch, even if it costs a bit more.
• James Urquhart with an information packed interview on a wide range of cloud topics: Cloud Expert Inside theCube at Stata Conference. Highlights: cloud is an operations model, it is not a technology, it is a way to apply technology to problems; the faster you can get the resources into the hands of the people who use it the more money you save overall; cloud is a cash flow story, not a savings story; services aren't about servers or storage, they are about applications.
• Quotable Quotes:
  • Ryan Tomayko: Frameworks don’t solve scalability problems, design solves scalability problems. Via @GregSkloot
  • @zuno: The golden rule of scalability: "it can probably wait" look for other areas to save resources.
  • @bihzad: joinserv hit a scalability wall, but I'm pretty sure I can climb over it with multiprocessing

Click to read more ...

Stuff The Internet Says On Scalability For February 4, 2011

Submitted for your reading pleasure...

• Super Bowl Prediction: Pittsburgh 27, Green Bay 24. I'll be rooting for Green Bay, but the Pittsburgh defense will eventually win the day, beating back the fleet footed, quick tossing, and sharp shooting Aaron Rodgers. Roethlisberger will make exactly 3 plays that matter, but they'll be the right 3 plays.
• Reddit is now at 1 billion page views a month. Congratulations!
• Amazon S3 Cloud Stores 262 Billion Objects.  My god, it's full of stars...
• Quora’s Technology Examined by Phil Whelan. Excellent detective work answering the question: How Does Quora Work?
• Quotable Quotes:
  • @timoreilly: When hardware became commoditized, software was valuable. Now that software being commoditized, data is valuable. #strataconf
  • @coldfusionPaul: "Write someone a query, they'll go away for a day. Teach someone to query, they'll just go away." so, I use #NoSQL 555
  • @squarecog: To go *really* fast, you want to get rid of spokes in your wheels, and ditch tires. Also, turning is overrated. #nosql

Click to read more ...

Stuff The Internet Says On Scalability For January 14, 2011

 Submitted for your reading pleasure...

• On the new year Twitter set a record with 6,939 Tweets Per Second (TPS). Cool video visualizing New Year's Eve Tweet data across the world. 
• Marko Rodriguez in Memoirs of a Graph Addict: Despair to Redemption tells a stirring tale of how graph programming saved the world from certain destruction by realizing Aritstotle's dream of an eudaimonia-driven society. Could a relational database do that?
• The never never ending battle of good versus evil has nothing on programmers arguing about bracket policies or sync vs async programming models. In this node.js thread, I love async, but I can't code like this, the battle continues. In the end programmers desire async, but leave the bar with sync.
• Quotable Quotes
  • @AmyDeLong: Walked into a starbucks and overheard 3 separate discussions all on scalability. #firstworldproblems #onlyinsf
  • @chvest: You may not need "high" scalability, but you should still consider your growth rates and prepare.

Click to read more ...

Stuff The Internet Says On Scalability For January 3, 2010

 Submitted for your reading pleasure...

• Quotable Quotes
  • @hofmanndavid: Performance and scalability anxiety makes developers want to catch the flying butterflies
  • @tivrfoa: "Scalability solutions aren't magic. They involve partitioning, indexing and replication." Twitter engineer
  • Alan Perlis: Fools ignore complexity; pragmatists suffer it; experts avoid it; geniuses remove it.
• CIO update: Post-mortem on the Skype outage. Interesting tale of a cascading collapse in complex, distributed, interactive systems. For more background see the highly illuminating Explaining Supernodes by Dan York.
• RethinkDB and SSD Databases. SSD was not a revolution by Kevin Burton. What’s really shocking to me, is that while SSD and flash storage is very exciting, it wasn’t as revolutionary in 2010 as I would have liked to have seen.
• The case for Datastore-Side-Scripting. Russell Sullivan predicts real-time web applications are going in the direction of being entirely event driven, from client (WebSockets) to web-server (Node.js) to datastore (Redisql). And to complete the even driven chain is datastore-side-scripting.
• Developments that could change everything...

  Click to read more ...

Stuff the Internet Says on Scalability For December 17th, 2010

• If you missed it here's a link to my webinar and here's the slidedeck for the talk with a buch of additional slides that I didn't have a chance to talk about. The funky picture of Lincoln is classic.
• Can MySQL really handle 1,000,000 req/sec? Sure, when you turn it into a NoSQLish database, skip all the SQL processing, and access the backend store directly. Percona is making this possible with their HandlerSocket plugin based on the work of Yoshinori Matsunobu.
• Quotable Quotes:
  • @labsji: If SQL is an abstraction of Big machines....NoSQL is an abstration of distributed computing.
  • @olov: man this eventual consistency #nosql thingy makes #facebook even more annoying. "you have a new comment, no you dont"
• Nice racks. Time has pictures of a Facebook datacenter. 

Click to read more ...

Stuff the Internet Says on Scalability For November 29th, 2010

Eating turkey all weekend and wondering what you might have missed?

• James Hamilton on why “all you have learned about disks so far is probably wrong" in Availability in Globally Distributed Storage. It turns out for the same reason our financial systems melt down: black swans. The world is predictably unpredictable. Murat Demirbas also has a good post on the same Google research paper.
• Stack Overflow Hits 10M Uniques
• Vroom...Formula One racecar streams 27 gigabytes of telemetry data during a race weekend! 200 sensors “measuring anything and everything that moves or gets warm. 
• Quotable Quotes:
  • @dmalenko: It is cool to sit by the ocean, oversee the sunset and think about scalability models for a web app
  • @detroitpro: I have to admit; sometimes I think "This would be easier with a SQL DB" #NoSQL #NotOften #ComplextRelationships #FindingRootObjects

Click to read more ...

Stuff the Internet Says on Scalability For November 12th, 2010

• Google – A Study In Scalability And A Little Systems Horse Sense. A nice summary by Krishna Sankar of a version of Jeff Dean's classic talk on Google Scalability given to Stanford's EE380 class.
• Quotable Quotes:
  • @jkalucki: Getting just 100 servers to work together for the first time is so ridiculously complicated. Horizontal scaling doesn't scale.
  • @simeons: Yahoo's scalability is drivem by lots of asynchronous processing. "You learn to love it." -- @rstata Yahoo's CTO

Click to read more ...

Hot Scalability Links For November 5th, 2010

So much good stuff this week...

• Adrian Cockcroft Compares NoSQL Availability Models. Let's risk feeding the CAP trolls, and try to get some insight into the differences between the many NoSQL contenders. Adrian asks how each NoSQL product will add a movie to its favorites list, read it back, and how this works across availability zones. Much trickier than it sounds with multiple writers. Cassandra and MongoDB answer back.
• Stuff the Internet Says:
  • @jerng: Reading up on scalability. WHY THE HELL FOR? Because I want to know the future.
  • @freerangedata: The #nosql options are the micro brews/craft beers of data stores. So many good ones, so little time to try them all.
  • @edward_ribeiro: Soon, Darwinism will start to play its role on #NoSQL systems. You know, only the fittest will survive.
  • @connectionreq: I'm always wowed when I hear how Facebook abuses their MySQL databases in crazy ways
  • @louismrose: This is the kind of scalability we should be working on... http://yfrog.com/59qb0oj
• Redis at Superfeedr. Each of our redis servers process on average 3500 queries per second.

Click to read more ...

Hot Scalability Links For Oct 24, 2010

On a cold and rainy Fall day, a day stolen from winter rather than our usual gorgeous Indian Summers, a day not even the SF Giants winning the pennant can help warm, here are some hot links to read by a digital flame: 

• Using MySQL as a NoSQL - A story for exceeding 750,000 qps on a commodity server by Yoshinori Matsunobu. Wonderfully detailed post on how you can lookup a row by ID really fast if you bypass all the typical MySQL query parsing overhead.
• Minecraftwiki.net and minecraftforum.net now serve more traffic than Slashdot and Stackoverflow! 1 million pageviews and 100k uniques  per day, per site; 10TB of bandwidth a month; 4+ machines running Varnish, HAProxy, PHP, MySQL, Nginx. 
• Stuff the Internet Says:
  • @old_sound: Somebody make me a t-shirt that says "I've read the CAP theorem and I liked it" 
  • @dscape: How relevant do I think the CAP theorem is? Not at all. I honestly hate conversations where anyone talks about crap.. cap, sorry. 
  • @humidbeing: If you hit limits of mysql, why reinvent the wheel by rolling your own solution when DBs like MS SQL and Oracle have proven scalability?
  • Ayende Rahien: You saved 5 cents, and your code is not readable, congrats!

  Click to read more ...

Hot Scalability Links For Oct 8, 2010

• So what happened at the Surge 2010 Conference? A lot...
  • Cosimo Streppone with a nicely detailed day by day summary.
  • Conference organizer Theo Schlossnagle says: Surge. Year one. Done. Awesome.
  • Robert Haas with SURGE Recap
  • From SEOmoz we have their SurgeCon Review – our thoughts on a fabulous web scalability conference
  • Scaling myYearbook.com
  • Availability, The Cloud and Everything by Joe Williams of Cloudant.
  • There's Plenty of Room at the Bottom. An Invitation to Explore with Network Flows by Benjamin Black.
• Quotable quotes for 200 Alex:
  • postwait: That which cannot be measured cannot be scaled.
  • gsylvest: Cameron Purdy:Instead of performance, think about scalability. Michael Nygaard:Instead of scalability, think of response time distributions
  • jkalucki: Sometimes this job is a bit too much like high-energy physics: Blast the invisible with a beam of hell fury, then decode the backscatter.
  • DrHayt: Shard for availability, not scalability. Do what is necessary to make sure that your shards to not share the same failure domain.

  Click to read more ...