Friday
Aug292008
Product: ScaleOut StateServer is Memcached on Steroids
Friday, August 29, 2008 at 2:39AM ScaleOut StateServer is an in-memory distributed cache across a server farm or compute grid. Unlike middleware vendors, StateServer is aims at being a very good data cache, it doesn't try to handle job scheduling as well.
StateServer is what you might get when you take Memcached and merge in all the value added distributed caching features you've ever dreamed of. True, Memcached is free and ScaleOut StateServer is very far from free, but for those looking a for a satisfying out-of-the-box experience, StateServer may be just the caching solution you are looking for. Yes, "solution" is one of those "oh my God I'm going to pay through the nose" indicator words, but it really applies here. Memcached is a framework whereas StateServer has already prepackaged most features you would need to add through your own programming efforts.
Why use a distributed cache? Because it combines the holly quadrinity of computing: better performance, linear scalability, high availability, and fast application development. Performance is better because data is accessed from memory instead of through a database to a disk. Scalability is linear because as more servers are added data is transparently load balanced across the servers so there is an automated in-memory sharding. Availability is higher because multiple copies of data are kept in memory and the entire system reroutes on failure. Application development is faster because there's only one layer of software to deal with, the cache, and its API is simple. All the complexity is hidden from the programmer which means all a developer has to do is get and put data.
StateServer follows the RAM is the new disk credo. Memory is assumed to be the system of record, not the database. If you want data to be stored in a database and have the two kept in sync, then you'll have to add that layer yourself. All the standard memcached techniques should work as well for StateServer. Consider however that a database layer may not be needed. Reliability is handled by StateServer because it keeps multiple data copies, reroutes on failure, and has an option for geographical distribution for another layer of added safety. Storing to disk wouldn't make you any safer.
Via email I asked them a few questions. The key question was how they stacked up against Memcached? As that is surely one of the more popular challenges they would get in any sales cycle, I was very curious about their answer. And they did a great job differentiation themselves. What did they say?
First, for an in-depth discussion of their technology take a look ScaleOut Software Technology, but here a few of the highlights:
- Development Edition: No Charge
- Professional Edition: $1,895 for 2 servers
- Data Center Edition: $71,995 for 64 servers
- GeoServer Option First two data centers $14,995, Each add'l data center $7,495.
- Support: 25% of software license fee
Some potential negatives about ScaleOut StateServer:
In the next section the headings are my questions and the responses are from ScaleOut Software.
Why use ScaleOut StateServer instead of Memcached?
I've [Dan McMillan, VP Sales] included some data points below based on our current understanding of the Memcached product. We don't use and haven't tested Memcached internally, so this comparison is based in part upon our own investigations and in part what we are hearing from our own customers during their evaluation and comparisons. We are aware that Memcached is successfully being used on many large, high volume sites. We believe strong demand for ScaleOut is being driven by companies that need a ready-to-deploy solution that provides advanced features and just works. We also hear that Memcached is often seen as a low cost solution in the beginning, but development and ongoing management costs sometimes far exceed our licensing fees.
What sets ScaleOut apart from Memcached (and other competing solutions) is that ScaleOut was architected from the ground up to be a fully integrated and automated caching solution. ScaleOut offers both scalability and high availability, where our competitors typically provide only one or the other. ScaleOut is considered a full-featured, plug-n-play caching solution at a very reasonable price point, whereas we view Memcached as a framework in which to build your own caching solution. Much of the cost in choosing Memcached will be in development and ongoing management. ScaleOut works right out of the box.
I asked ScaleOut Software founder and chief architect, Bill Bain for his thoughts on this. He is a long-time distributed caching and parallel computing expert and is the architect of ScaleOut StateServer. He had several interesting points to share about creating a distributed cache by using an open source (i.e. build it yourself) solution versus ScaleOut StateServer.
First, he estimates that it would take considerable time and effort for engineers to create a distributed cache that has ScaleOut StateServer's fundamental capabilities. The primary reason is that the open source method only gives you a starting point, but it does not include most capabilities that are needed in a distributed cache. In fact, there is no built-in scalability or availability, the two principal benefits of a distributed cache. Here is some of the functionality that you would have to build:
In addition to the above, we hope that the fact ScaleOut Software provides a commercial solution that is reasonably priced, supported and constantly improved would be viewed as an important plus for our customers. In many cases, in-house and open source solutions are not supported or improved once the original developer is gone or is assigned to other priorities.
Do you find yourself in competition with the likes of Terracotta, GridGain, GridSpaces, and Coherence type products?
Our ScaleOut technology has previously been targeted to the ASP.Net space. Now that we are entering the Java/Linux space, we will be competing with companies like the ones you mentioned above, which are mainly Java/Linux focused as well.
We initially got our start with distributed caching for ecommerce applications, but grid computing seems to be a strong growth area for us as well. We are now working with some large Wall Street firms on grid computing projects that involve some (very large) grid operations.
I would like to reiterate that we are very focused on data caching only. We don't try to do job scheduling or other grid computing tasks, but we do improve performance and availability for those tasks via our distributed data cache.
What architectures your customers are using with your GeoServer product?
A. GeoServer is a newer, add-on product that is designed to replicate the contents of two or more geographically separated ScaleOut object stores (caches). Typically a customer might use GeoServer to replicate object data between a primary data center site and a DR site. GeoServer facilitates continuous (async.) replication between sites, so if site A goes offline, the other site B is immediately available to handle the workload.
Our ScaleOut technology offers 3 primary benefits: Scalability, performance & high availability. From a single web farm perspective, ScaleOut provides high availability by making either 1 or 2 (this is configurable) replica copies of each master object and storing the replica on an alternate host server in the farm. ScaleOut provides uniform access to the object from any server, and protects the object in the case of a server failure. With GeoServer, these benefits are extended across multiple sites.
It is true that distributed caches typically hold temporary, fast-changing data, but that data can still be very critical to ecommerce, or grid computing applications. Loss of this data during a server failure, worker process recycle or even a grid computation process is unacceptable. We improve performance by keeping the data in-memory, while still maintaining high availability.
Reader Comments (18)
Is this an article or a pruduct placement?
Check out the Microsoft Velocity project. It's nearing CTP2 and looks like it will be bigger, faster, strong, and probably cheaper than scaleout.
I thought that Brad Fitzpatrick, ex-Six Apart, and now of Google, maintains a free, proven (used by MT) sharded and mechached codebase? No?
Yes, here you go, the man is a force. Although from what I read here, the price aint bad for a real out of box system with support and features for the mid-range shop uncomfortable with Open Source.
http://www.danga.com/
There is repcached which is memcached + replication if I recall.
also, client implementations such as the PHP PECL memcache client can support sending data to multiple servers, which can provide the redundancy mentioned above.
> Is this an article or a product placement?
Velocity is definitely on my list. Is there something in particular you disagree with?
Performance: Scales with measured linear throughput gain to farms with 64 servers. StateServer was subjected to maximum access load in tests that ramped from 2 to 64 servers, with more than 2.5 gigabytes of cached data and a sustained throughput of over 92,000 accesses per second using a 20 Mbits/second Infiniband network. StateServer provided linear throughput increases at each stage of the test as servers and load were added.
Haha, that paragraph made me laugh out loud.
Someone invite that marketing guy to 2008, will ya?
"2.5 GB of cached data"?
"92k reads/sec"?
"20Mbit/s Infiniband link"???
I mean, I have seen bad product placements but this really takes the cake. If you want to amuse a technical audience then at least get your numbers right - or don't even bother to try if these numbers are real.
2.5G of cached data is a joke. I keep over 6 times that much in a single memcached instance.
92k reads/sec sounds impressive at a glance but is absolutely meaningless when you don't tell anything about the nature of these reads (size, locality?). I call snakeoil.
20Mbit/s Infiniband? Dude, someone really ripped you off there. But hey I can fix that for you. For only a few thousand bucks I'll deploy a high-end 10Mbit/s GBit Ethernet infrastructure for you. Just give me a call.
> Just give me a call.
Since you are Anonymous that will be difficult.
The size of the cached data seems small to me too, but those are the numbers they report. Make of them what you will.
"We don't use and haven't tested Memcached internally, so this comparison is based in part upon our own investigations and in part what we are hearing from our own customers during their evaluation and comparisons." <-- this is where i called bullshit. this is like saying you are building a commercial search engine and you've never used google. whatever.
> There is repcached
Thanks. I hadn't seen that before. The link is http://repcached.lab.klab.org/ . "repcached 2.0 supports master-master replication. 2.0's replication cooperate with client library's server balancing feature."
Ugh.
This reads as though they read every "I wish for" e-mail on the memcached mailing list list and went to implement them. Then they advertised that all of these features are "free", and give numbers which are pathetically small.
Memcached replication is trivial... you do it in the client. libmemcached has tentative internal support for this, as do other clients.
Rebalance objects? Yawn.
Search based on metadata? I'm *sure* that works fine.
There are generally *really good reasons* why these features don't exist core to memcached. We're not being stubborn so much as we understand at what point you're just doing it wrong. Calling this thing a "high scalability" product is almost criminal, since I guarantee it wouldn't survive a beta round in my hands, nor could it run any of my sites. They're stealing money from people who don't need a "solution" as much as they need to understand data caching better.
Also, holy shit, telling people their data is safe because you put *two* copies of it in RAM? I've been through HOW many datacenter power outages now?
> There are generally *really good reasons* why these features don't exist core to memcached.
I would find helpful if you could please explain the reasoning. And I believe the safety story is more the geographical distribution to multiple datacenters.
A lot of it's in the FAQ (since these are frequent questions). The others have been usually answered exhaustively on the mailing list.
A summary of that, perhaps later transcribed to the FAQ, is something on my list to write. I'll plan to post this soon.
The article mentioned the local rebalancing and replication of data several times, and completely separately, of their multidatacenter bits. Given how many of their clients are unlikely to use multiple datacenters, I'd be extremely wary of anyone suggesting that as a datastore.
> is something on my list to write. I'll plan to post this soon.
Having made vigorous objection it would be quite convenient of you could make a brief form of your case here. Because I've read the FAQ numerous times and I'm not sure what you are specifically referring to. And I'm not sure if you are stating the because something shouldn't be in core memcached that it shouldn't be anywhere else either.
> he article mentioned the local rebalancing
Rebalancing is quite a nice feature. I'm surprised you would be against it. It allows the transparent sharding and elastic partitioning. Why do you think this is a poor feature?
> Given how many of their clients are unlikely to use multiple datacenters
Most people are in catch-22. You can't use multiple datacenters because the software is hard to write. If the middleware layers handle it, as does other software like GigaSpaces, we might see a lot more multi-datacenter action, especially as more clouds come on-line and we see that they do indeed fail.
> Having made vigorous objection it would be quite convenient of you could make a brief form of your case here
It's most definitely in the mail archives, but I've agreed that's not the best place. There's a long list of them and I'd rather put it out more formally. I pay attention to every idea people mention. Good, bad, or ugly.
The really short answer is they shouldn't be in memcached, either at all, or in these specific forms. The rest are things databases do. As you add these "fancy" features, actually using those commands removes the linear scalability of the service. Like fetch by tag, fetch by index, etc.
> Rebalancing is quite a nice feature. I'm surprised you would be against it. It allows the transparent sharding and elastic partitioning. Why do you think this is a poor feature?
That insinuates there's an index being used somewhere, which is wrong wrong wrong. There're a few other ways to do it I guess... but all of them require either guesswork or maintaining a map of metadata.
MogileFS rebalances and changes what paths it gives you for good reasons (IO overload, multiple datacenters, storage hosts dying...), but mogilefs is designed for managing persistent storage, so it requires a metadata index, which in reality is a database instance.
"hot objects" in memcached don't really exist since all objects are returned in constant time. I do also advocate usage of local cache for those few objects you hit on *every* page load, and can deal with a 5 minute refresh time. However that's distinctively *not* linked into something that memcached expires for you (another bullet point in the article). Since that also suggests broadcast communication of expiration, which isn't linear.
That's just overcomplicating the whole deal anyway. A *failure* should increase partial cache misses. Then replacing the server, adding new servers, removing servers, really hasn't been an issue since consistent hashing was made popular. Adding a server causes a few percent extra cache misses. A memcached cluster is already pretty darn elastic.
> Most people are in catch-22. You can't use multiple datacenters because the software is hard to write. If the middleware layers handle it, as does other software like GigaSpaces, we might see a lot more multi-datacenter action, especially as more clouds come on-line and we see that they do indeed fail.
This is something I'd agree on as needing more work. That's a *very* narrow list compared to the rest of this blatant product advertisement. Multidatacenter memcached works okay now (if you use libmemcached, and mysql with the libmemcached-based UDF's... or one of the proxies no one's bothered releasing, or a mysql proxy... blah blah). Yeah, it's hard. But a lot of other things about multiple datacenter space is actually pretty hard. I'd still insist that very few companies will ever do it, and if so only if a "cloud" gives it to them for free.
Fundamentally the article is describing something which cannot exist in a large scale, or with many clients, either by its advertisement of broadcast events, or by using scanning and mass effect commands. If you use none of these features it might scale as well as memcached, but it might as well *be* memcached.
Excellent. I appreciate the effort. That helps a lot.
I think the key may be when you say "The rest are things databases do" and "as much as they need to understand data caching better."
These features go beyond caching and are part of a move to in-memory grid type systems. For StateServer behaviour is separated from the data layer whereas as in GigaSpaces behaviour and data are colocated, but in both cases the database is largely redundant. So if that's no the type of system you want to build I can see where the objections come from.
> hanges what paths it gives you for good reasons (IO overload, multiple datacenters, storage hosts dying...
SLA driven reconfiguration would be a good feature.
> but mogilefs is designed for managing persistent storage
What if RAM is the persistence storage?
> However that's distinctively *not* linked into something that memcached expires for you (
Wouldn't it be better to have local caching implemented once and tied in with memcached so they stay consistent?
> A *failure* should increase partial cache misses.
Certainly makes sense for a pure data cache. Cameron Purdy has a take that may make more sense for in-memory system: "Consistent hashing, while a step forward from the hashing performed by Memcached (etc.), is not the state of the art. Dynamic partitioning provides all of the benefits of consistent hashing, yet allows for a smooth (lossless) transition from one cluster view (specific set of members) to another, with no loss of data."
http://www.artima.com/forums/flat.jsp?forum=276&thread=219807
> (if you use libmemcached, and mysql with the libmemcached-based UDF's
This is way too complex an approach. Facebook, Fotolog, and few dedicated others may be able to pull it off, but it's a lot to ask. And it again puts the database as the system of record. When you remove that requirement you get a different sort of architecture.
> That's a *very* narrow list compared to the rest of this blatant product advertisement.
In the sense of listing features a product thinks makes them distinctive. But if you are insinuating anything else you are offensively off base.
I think the base of my issue with the advertisement is that they're really talking about a database, and saying it's a "souped up memcached" is a horrible misnomer. Yes, a database will absolutely have more index-related features than something designed as a cache. That's also insinuating that memcached suffers by explicitly missing these features. So comparing the two directly, *especially* when declaring mid-article that they've never investigated its proper usage, might elicit unrequested name calling.
Memcached is designed to try as hard as it can to ensure you have fast access to relevant data. The generic memcached design pattern is to maximize the global size of the cache (adding servers gives a linear addition to capacity), while being utilized so only your active dataset needs to be in RAM (rule of thumb having that value stick around 10-20% of your full dataset). Something that replicates all of your data cuts the number of active objects you can store in half. The benefit of being able to store *twice* as many items in cache, for anything I work on, far outweighs the benefit of mitigating a small percentage of cache misses. I'm aware there are people for which this isn't true, which is why some memcached clients do support replication.
I'll go into more detail in a blog post. There's a list :P
> SLA driven reconfiguration would be a good feature.
"Someone's SLA" kinda describes mogilefs' existing features. We've been slowly working on making that all pluggable and easy to deal with. MogileFS 2.20 supports multiple datacenters via a plugin on its replication policy. No, none of that's documented, I'm working on it this weekend :)
> What if RAM is the persistence storage?
I don't really believe in that argument.. I'm a DBA by trade and my focus is to ensure data doesn't go away. Part of that is to explicitly not put the *only* copies of any data into anything that's not backed by magnetic media and redundant by host. Otherwise, I think mogilefs will actually work fine if you give it devices that are tmpfs filesystems ;) I was playing around with giving mogilefs a special file class of "unreliable + fast" - which would store the item only in RAM, but make a best effort attempt at keeping it alive via its replication policies. This isn't entirely useful for all objects due to filesystem cache dealing with read load, but great for large temporary objects with high churn, or ones which can be easily regenerated.
> Wouldn't it be better to have local caching implemented once and tied in with memcached so they stay consistent?
Part of the reason why memcached exists is because doing that at scale is really hard. Expiring an item in memcached means hashing to the server and sending a 'delete' command to that box. Active expiration for a distributed *local* cache requires broadcast. (ie; any system where data is stored more than once).
> Certainly makes sense for a pure data cache. Cameron Purdy has a take that may make more sense for in-memory system:
That's assuming replication.. Dynamic partitioning might also be alluding to the virtual buckets idea that was half implemented in 2004. The cache is hashed into, say, 1024 buckets, then your keys are hashed out onto that, and the buckets are dynamically moved between servers by a management process (yay, centralized metadata server..). Unsurprisingly, there hasn't been a huge draw to complete this feature...
> This is way too complex an approach
Yup... that was my point. It's doable but it's too hard.
Although Velocity has made progress from CTP1 to CTP2, it still leaves much to be desired. It will be some time before they provide all the important features in a distributed cache and even longer before it is tested in the market. I wish them good luck.
In the meantime, NCache already provides all CTP2 & V1, and many more features. NCache is the first, the most mature, and the most feature-rich distributed cache in the .NET space. NCache is an enterprise level in-memory distributed cache for .NET and also provides a distributed ASP.NET Session State. Check it out at http://www.alachisoft.com">Distributed Cache.
NCache Express is a totally free version of NCache. Check it out at http://www.alachisoft.com/rp.php?dest=/ncache/ncache_express.html">Free Distributed Cache.
If you are interested in performance numbers of all the known distributed caching solutions available then check out
http://operationtimeout.wordpress.com
I'm currently researching into what options there are in terms of caching a complex financial system. And it very much looks like StateServer is still nowhere near competing GemFire and the likes in terms of features, the most important missing one being persistence to disk for durability (rather than to a shared bottleneck like a DB).
Documentation is also rather sparse, which makes evaluation harder. MS AppFabric Caching does not persist to disk either and has a very limited set of features. The one remaining option on the Windows/.NET side of things I still have to check out is NCache, but if that turns out to be similarly limited, there wouldn't be anything that could even remotely compete with the variety of Java products out there for many years now. I would be glad to be proven wrong.
For real applications (rather than read-mostly, memcached-able ones) that need to be scalable, durable caches that don't fall back to a DB are an absolute necessity.