Shutterfly Saw a Speedup of 500% With Flashcache
Thursday, January 5, 2012 at 9:15AM In the "should I or shouldn't I" debate around deploying SSD, it always helps to have real-world data. Fiesta! with a live-blog summary of a presentation by Kenny Gorman on Shutterfly on MongoDB Performance Tuning.
What if you still need more performance after doing all of this tuning? One option is to use SSDs. Shutterfly uses Facebook’s flashcache: kernel module to cache data on SSD. Designed for MySQL/InnoDB. SSD in front of a disk, but exposed as a single mount point. This only makes sense when you have lots of physical I/O. Shutterfly saw a speedup of 500% w/ flashcache. A benefit is that you can delay sharding: less complexity.
The whole series of posts has a lot of great information and is worth a longer look, especially if you are considering using MongoDB.
Related Articles
- Slides for MongoSF 2011 slides: MongoDB Performance Tuning
- SSD+HDD sharding setup for large and permanently growing collections
- Imlementing MongoDB at Shutterfly by Kenny Gorman
- MongoSV follow up
- MongoSV 2011 Conference Vidoes
- Sharing Life’s Joy using MongoDB: A Shutterfly Case Study
- Real World NoSQL: MongoDB at Shutterfly
- Scale-Out Vs Scale-Up
Reader Comments (2)
Not sure the benefit is compelling. Yes, you reduce complexity by avoiding sharding, but you add considerable complexity by adding another caching layer and having to deal with and debug cache consistency issues.
What would be ideal is to compare the cost of their SSD cache to the cost of other choices that might yield even higher speedups. A x5 speedup is a big speedup, but probably is just from avoiding hitting disk a lot, and adding memory to existing layers might have had a similar or larger effect.
For some workload you can't rely on adding memory to cache your way out of performance issues without resorting to having the entire data set fit in memory. At some point it is cheaper to scale out random IO using flash then it is to fit the entire set in memory because the amount of random IO far exceeds what any spinning disk will give you no matter how much data you cache. You could have the hottest %95 of the entire data set in memory and still exceed the random IO capacity of a spinning disk array.