Strategy: Don't Use Polling for Real-time Feeds

Ivan Zuzak wrote a fascinating article on Real-time feed processing and filtering using Google App Engine to build Feed-buster, a service that inserts MediaRSS tags into feeds that don't have them. He talks about using polling and PubSubHubBub (real-time) to process FriendFeed feeds. Ivan is trying to devise a separate filtering service where: 

1. filtering services should be applied as close to the publisher as possible so notifications that nobody wants don’t waste network resource.
2. processing services should be applied as close to the subscriber so that the original update may be transported through the network as a single notification for as long as possible.

Besides being a generally interesting article, Ivan makes an insightful observation on the nature of using polling services in combination with metered Infrastructure/Platform services:

Polling is bad because AppEngine applications have a fixed free daily quota for consumed resources, when the number of feeds the service processed increased - the daily quota was exhausted before the end of the day because FF polls the service for each feed every 45 minutes.

This fits directly in with the ideas in Cloud Programming Directly Feeds Cost Allocation Back into Software Design. My general preference is to poll a distributed queue for work items. It's robust and allows your system to control it's own resource usage by determining when to poll. Otherwise you can easily be overwhelmed by fast pushers. Here the overwhelming is going the other way. Your budget is being overwhelmed by the polling requests. And the more you try approximate real-time with frequent polling requests the more your budget is busted.

It's a cool example of how costs, algorithm, and platform choices all feed into and shape product architectures.

Have We Reached the End of Scaling?

This is an excerpt from my article Building Super Scalable Systems: Blade Runner Meets Autonomic Computing in the Ambient Cloud.

Have we reached the end of scaling? That's what I asked myself one day after noticing a bunch of "The End of" headlines. We've reached The End of History because the Western liberal democracy is the "end point of humanity's sociocultural evolution and the final form of human government." We've reached The End of Science because of the "fact that there aren't going to be any obvious, cataclysmic revolutions." We've even reached The End of Theory because all answers can be found in the continuous stream of data we're collecting. And doesn't always seem like we're at The End of the World?

Motivated by the prospect of everything ending, I began to wonder: have we really reached The End of Scaling?

Click to read more ...

11 Strategies to Rock Your Startup’s Scalability in 2010

This is a guest posting by Marty Abbott and Michael Fisher, authors of The Art of Scalability. I'm still reading their book and will have an interview with them a little later.

If 2010 is the year that you’ve decided to kickoff your startup or if you’ve already got something off the ground and are expecting double or triple digit growth, this list is for you.  We all want the attention of users to achieve viral growth but as many can attest, too much attention can bring a startup to its knees.  If you’ve used Twitter for any amount of time you’re sure to have seen the “Fail Whale”, which is so often seen that it has its own fan club.  Take a look at the graph below from Compete.com showing Twitter’s unique visitors.  One can argue that limitations in the product offering have as much to do with the flattening of growth over the past six months as does the availability, but it’s hard to believe the inability of users to actually use the service has not hindered growth.

What should you do if you want your startup to scale with double and triple digit growth?  We’ve put together a list of 11 strategies that will aid in your quest for scalability.  In our recently released book “The Art of Scalability” you will find more details about these and other strategies.

Click to read more ...

Terrastore - Scalable, elastic, consistent document store.

Terrastore is a new-born document store which provides advanced scalability and elasticity features without sacrificing consistency.

Here are a few highlights:

• Ubiquitous: based on the universally supported HTTP protocol.
• Distributed: nodes can run and live everywhere on your network.
• Elastic: you can add and remove nodes dynamically to/from your running cluster with no downtime and no changes at all to your configuration.
• Scalable at the data layer: documents are partitioned and distributed among your nodes, with automatic and transparent re-balancing when nodes join and leave.
• Scalable at the computational layer: query and update operations are distributed to the nodes which actually holds the queried/updated data, minimizing network traffic and spreading computational load.
• Consistent: providing per-document consistency, you're guaranteed to always get the latest value of a single document, with read committed isolation for concurrent modifications.
• Schemaless: providing a collection-based interface holding JSON documents with no pre-defined schema, you can just create your collections and put everything you want into.
• Easy operations: install a fully working cluster in just a few commands and no XML to edit.
• Features rich: support for push-down predicates, range queries and server-side update functions.

Read, participate, download and clone it!

Zynga Needs a Server-side Systems Engineer

Ashleigh Anderson from Zynga let me know that they have an opening for a Systems Engineer working on some new games they are developing. Given the state of the job market I thought it worth posting. Here are more details...

Click to read more ...

Incremental deployment

Incremental deployment. Manual/automated hybrid deployment strategy we use at Forward for our critical, high availability distributed systems.

Hot Holiday Scalability Links for 2009

• Building Super Scalable Systems: Blade Runner Meets Autonomic Computing in the Ambient Cloud. The only independent platform most of us will have access to capable of hosting planet-scale applications is the Ambient Cloud. It forms a sort of digital potluck where everyone contributes memory, network, and other compute resources from whatever they happen to have available. 
• Top 10 Internet Startup Scalability Killers. Strategies taken from The Art of Scalability. 1. Thinking Scalability Is Just About Technology; 2. Overuse of Synchronous Calls; 3. Failure to Weed or Seed Soon Enough; 4. Inappropriate Use of Databases; 5. Cesspools Instead of Swim Lanes; 6. Reliance on Vertical Scale; 7. Failure to Learn from History; 8. Changing Development Methodologies to Fix Problems; 9. Too Little Caching, Too Late; 10. Overreliance on Third Parties to Scale.
• The New Google: Internet Giant Opens Up About Real-Time and Local Search, Cloud Computing, and Data Liberation. In four separate interviews, Google delved into some of the most important topics of the day, from its advances in real-time and local search to cloud computing and a “data.. liberation” effort to help consumers export their files and digital information from Google products 
• Ask HN: What are the best technologies you've worked with this year? Quite a nice variety, no clear winner. Some honorable mentions: Django, Redis, Clojure, XMPP, Node.js, AMQP, Rails, jQuery, Solr, Hadoop.
• Why I think Mongo is to Databases what Rails was to Frameworks. We have been amazed at how much code we cut out of Harmony with the switch from MySQL to Mongo. 
• A Deluge of Data Shapes a New Era in Computing. Dr. Gray called the shift a “fourth paradigm.” The first three paradigms were experimental, theoretical and, more recently, computational science. He explained this paradigm as an evolving era in which an “exaflood” of observational data was threatening to overwhelm scientists.
• MySpace Replaces Storage with Solid-State Drive Technology in 150 Standard Load Servers. Using SSD reduced the headcount of their heavy load servers from 80 to 30.
• Amazon's CloudFront Now Offers Flash Streaming, This Will Disrupt The Market. Amazon does have the potential to take many of the mid-sized customers who spend between $3-5k a month on video. 
• How would you design an AppEngine datastore for a social site like Twitter? Using Jaiku's reimplementation on Google App Engine is a good reference.
• Query Processing for NOSQL DB. It seems to me that the responsibility of building an indexing and query mechanism lands on the NoSQL user.
• Persistent Trees in git, Clojure and CouchDB. There are some really neat software projects emerging at the moment, and as a developer I always find it interesting to take a look at the implementation details, because there is often a lot to be learned. 
• Transcendent Memory. Transcendent memory is a new memory-management technique which, it is hoped, will improve the system's use of scarce RAM, regardless of whether virtualization is being used.
• Trading Shares in Milliseconds. By the end of the day, his computers will have bought and sold about 60 million to 80 million shares.

Oracle and IBM databases: Disk-based vs In-memory databases

Current disk based RDBMS can run out of steam when processing large data. Can these problems be solved by migrating from a disk based RDBMS to an IMDB? Any limitations? To find out, I tested one of each from the two leading vendors who together hold 70% of the market share - Oracle's 11g and TimesTen 11g, and IBM's DB2 v9.5 and solidDB 6.3.

read more at BigDataMatters.com

The most common flaw in software performance testing

How many times have we all run across a situation where the performance tests on a piece of software pass with flying colors on the test systems only to see the software exhibit poor performance characteristics when the software is deployed in production? Read More Here...

Building Super Scalable Systems: Blade Runner Meets Autonomic Computing in the Ambient Cloud

"But it is not complicated. [There's] just a lot of it." 
-- Richard Feynman on how the immense variety of the world arises from simple rules.

Contents:

1. Have We Reached the End of Scaling?
2. Applications Become Black Boxes Using Markets to Scale and Control Costs
3. Let's Welcome our Neo-Feudal Overlords
4. The Economic Argument for the Ambient Cloud
5. What Will Kill the Cloud?
6. The Amazing Collective Compute Power of the Ambient Cloud
7. Using the Ambient Cloud as an Application Runtime
8. Applications as Virtual States
9. Conclusion

We have not yet begun to scale. The world is still fundamentally disconnected and for all our wisdom we are still in the earliest days of learning how to build truly large planet-scaling applications.

Today 350 million users on Facebook is a lot of users and five million followers on Twitter is a lot of followers. This may seem like a lot now, but consider we have no planet wide applications yet. None.

Tomorrow the numbers foreshadow a new Cambrian explosion of connectivity that will look as different as the image of a bare lifeless earth looks to us today. We will have 10 billion people, we will have trillions of things, and we will have a great multitude of social networks densely interconnecting all these people to people, things to things, and people to things.

How can we possibly build planet scalable systems to handle this massive growth if building much smaller applications currently stresses architectural best practices past breaking? We can't. We aren't anywhere close to building applications at this scale, except for perhaps Google and a few others, and there's no way you and I can reproduce what they are doing. Companies are scrambling to raise hundreds of millions of dollars in order to build even more datacenters. As the world becomes more and more global and more and more connected, handling the load may require building applications 4 or 5 orders of magnitude larger than any current system. The cost for an infrastructure capable of supporting planet-scale applications could be in the 10 trillion dollar range (very roughly estimated at $100 million a data center times 10K).

If you aren't Google, or a very few other companies, how can you possibly compete? For a glimmer of a possible direction that may not require a kingdom's worth of resources, please take a look at this short video:

Click to read more ...