Monday
Sep102007
Is there a difference between partitioning and federation and sharding?
Monday, September 10, 2007 at 1:40AM
Unlike Theo Schlossnagle, author of Scalable Internet Architectures, I am not a stickler for semantics because I have an unswerving faith in the ultimate unknowability of the world as experienced by others. That's why it is Theo who bravely tackles the differences in his informative blog post Partitioning vs. Federation vs. Sharding. Royans Tharakan also talks about it on his blog.
Is there a difference and does it really matter to all our intrepid scalable website builders?
Generally whatever Theo says is probably close to the truth. Yet, in my mind I think of partitioning as a basic level category and federation and sharding as more specific (subordinate) instances of partitioning. And partitioning is a more specific instance of the more more general (superordinate) category divide-and-conquer. Which isn't a useful way to think about the topic at all.
So, let's say federation is like Star Trek. The Vulcans, Klingons, and Humans live in very separate policy domains, but they each pledge to work together to make sure Captain Kirk always gets the girl. And sharding is like AJAX, a great marketing term for stuff that may have already existed, but has taken on a new useful life on its own.
And that new useful life is that there are very specific examples of how sharding works, how it has been successful for existing web sites, and how you can create your scalable web site use shards. Federation and partitioning are far more nebulous less pragmatic concepts, so I am more than happy to AJAXify sharding into the popular lexicon :-)
Related Articles:
An Unorthodox Approach to Database Design : The Coming of the Shard.
Is there a difference and does it really matter to all our intrepid scalable website builders?
Generally whatever Theo says is probably close to the truth. Yet, in my mind I think of partitioning as a basic level category and federation and sharding as more specific (subordinate) instances of partitioning. And partitioning is a more specific instance of the more more general (superordinate) category divide-and-conquer. Which isn't a useful way to think about the topic at all.
So, let's say federation is like Star Trek. The Vulcans, Klingons, and Humans live in very separate policy domains, but they each pledge to work together to make sure Captain Kirk always gets the girl. And sharding is like AJAX, a great marketing term for stuff that may have already existed, but has taken on a new useful life on its own.
And that new useful life is that there are very specific examples of how sharding works, how it has been successful for existing web sites, and how you can create your scalable web site use shards. Federation and partitioning are far more nebulous less pragmatic concepts, so I am more than happy to AJAXify sharding into the popular lexicon :-)
Related Articles:
Reader Comments (2)
Great article. However, I disagree on two points.
1. As Theo says, the federation is really a delegation of the data partition. It is a significantly different enough that it deserves its own classification. To classify it under partitioning would muddy the already muddy waters.
2. I believe calling partitioning and federation nebulous and less pragmatic is incorrect and hard to understand. As Theo says, all of these concepts have been around for sometime.
These concepts have been around in the architecture world for quite sometime. However, the knowledge of these methodologies has been confined to architecture discussions and general IT populace hasn't had a thorough understanding of these. Further, there were no popular tools to manage the federation and sharding that a company in its infancy could deploy by getting a product off the shelves.
It is only recently that the availability of the knowledge and tools has made the partitioning/federation/sharding much more cost-effective and cost-effective proposition. That is a strong reason to clearly define these teminologies and need to be brought into popular lexicon with meaningful names.
Theres a lot to tell on each term specefied in the topic.
-----
http://underwaterseaplants.awardspace.com">sea plants
http://underwaterseaplants.awardspace.com/seagrapes.htm">sea grapes...http://underwaterseaplants.awardspace.com/seaweed.htm">seaweed