IBMer Says LAMP Can't Scale
Tuesday, December 25, 2007 at 12:49AM A very entertaining and somewhat educational article on IBM Poopheads say LAMP Users Need to "grow up". The physical three tier architecture turns out to be the root of all evil and shared nothing architectures brings simplicity and light.
In the comments Simon Willison makes an insightful comment on why fine grained caching works for personalized pages and proxy's don't:
Great post, but I have to disagree with you on the finely grained caching part. If you look at big LAMP deployments such as Flickr, LiveJournal and Facebook the common technology component that enables them to scale is memcached - a tool for finely grained caching. That's not to say that they aren't doing shared-nothing, it's just that memcached is critical for helping the database layer scale. LiveJournal serves around 50% of its page views "permission controlled" (friends only) so an HTTP proxy on the front end isn't the right solution - but memcached reduces their database hits by 90%.
Reader Comments (6)
Memcached is a great solution for fine-grained caching, but it really only masks the scaling problem. It fixes the symptom. The "M" in the LAMP is really the problem with scaling. MySQL does not scale. Period. Monolithic SQL databases don't scale in any real way, no matter how you cluster them. Until people deal with this there won't be a readily available, scalable shared nothing service out there that isn't built on a massive scale (see Google, Yahoo's search engine, Amazon, etc).
The real key to my argument is that the modern SQL database is more or less useless for web serving at scale. Tuning queries and turning server and database performance knobs will still be the headache that it is today. Considering many of the current frameworks in use (Cake, RoR, TurboGears) use the database as a dumb data store I wonder why they don't write their own abstraction starting with memcached as the source and adding some sort of efficient storage/persistence layer. If you're querying a key/value store for > 50% of your data, why not use it for everything instead of relying on something that wasn't designed to be used for the web.
Memcached is a nice tool. However it's something that requires a lot of error-prone manual configuration and tuning. A good application framework helps by making it easy to have a multi-layered caching strategy, e.g.
http://docs.jboss.org/seam/2.0.0.GA/reference/en/html/cache.html">JBoss Seam.
Aaron,
That's pretty much the premise behind the commercial products GigaSpaces and Coherence. Both of them provide very fast access to key-value tuples.
There's some inertia created at the next layer up, though. Current app server frameworks assume a relational model for the data store. It would be interesting to see a persistence layer that worked directly with one of these distributed memory grids. (As the primary data store, rather than either a cache on the side or a write-through SQL proxy.)
Michael T. Nygard
michael@michaelnygard.com
http://www.michaelnygard.com/
Author of "Release It!"
http://pragmaticprogrammer.com/titles/mnee/index.html
Amazon's SimpleDB is right in line with what Aaron describes. I'm curious to see if it causes any more move away from relational databases for simple systems deployed at very large scales.
Facebook uses FreeBSD on the backend. For memcached they use several hundred 64bit 6.2 machines with 16GB of ram per server. Irrespective of the OS, memcached is a highly vaulable piece when you need to scale. I agree with Aaron Huslage about how the traditional SQL DB holds one back on REAL scaling on demand.
Lamp - suxx