Friday
Feb222008
Kevin's Great Adventures in SSDland
Friday, February 22, 2008 at 1:53PM
Update: Final Thoughts on SSD and MySQL AKA Battleship Spinn3r. Tips on how to make your database 10x faster using solid state drives. Potential exists for 100x speedup.
Solid-state drives (SSDs) are the holy grail of storage. The promise of RAM speeds and hard disk like persistence have for years driven us crazy with power user lust, but they've stayed tantalizingly just out of reach. Always too expensive, too small, and oddly too slow. Has that changed? Can you now miraculously have your cake and eat it too? Can you now have it both ways? Is balancing work with family life now as easy as tripping over a terabyte drive?
In a pioneering series of blog articles Kevin Burton conducts original research on next generation SSD drives in real world configurations. For an experience report on his great adventure you can turn to:
Could SSD Mean a Rise in MyISAM Usage?, Serverbeach, MySQL and Mtron SSDs, Prediction: SSD Blades in 2008, Zeus IOPS - Another High Performance SSD, Thoughts on SSD and MySQL 5.1, Thoughts on Maria and SSD, 24 Hours with an SSD and MySQL, Random Write Performance in SSDs, SSD + PBXT = Crazy Suspicious!, More SSD vs HDD vs InnoDB vs MyISAM Numbers.
A lot of fascinating findings so far. Unfortunately Goldilocks may still find the porridge too slow. SSDs turn out to be fast, but not as fast as you might hope in out of the box configurations. MySQL on SSD is fast for sequential reads and writes, but random reads are writes are relatively slow. Kevin speculates "Log structured filesystems can come into play here and seriously increase performance by turning all random writes into sequential writes" and that "Bigtable and append only databases would FLY on flash."
The upshot for me is we need a storage engine designed specifically for SSDs as they provide a very different design space from hard disks. Kevin has a lot of excellent details and observations on his site and he'll no doubt be coming up with a lot more.
Solid-state drives (SSDs) are the holy grail of storage. The promise of RAM speeds and hard disk like persistence have for years driven us crazy with power user lust, but they've stayed tantalizingly just out of reach. Always too expensive, too small, and oddly too slow. Has that changed? Can you now miraculously have your cake and eat it too? Can you now have it both ways? Is balancing work with family life now as easy as tripping over a terabyte drive?
In a pioneering series of blog articles Kevin Burton conducts original research on next generation SSD drives in real world configurations. For an experience report on his great adventure you can turn to:
Could SSD Mean a Rise in MyISAM Usage?, Serverbeach, MySQL and Mtron SSDs, Prediction: SSD Blades in 2008, Zeus IOPS - Another High Performance SSD, Thoughts on SSD and MySQL 5.1, Thoughts on Maria and SSD, 24 Hours with an SSD and MySQL, Random Write Performance in SSDs, SSD + PBXT = Crazy Suspicious!, More SSD vs HDD vs InnoDB vs MyISAM Numbers.
A lot of fascinating findings so far. Unfortunately Goldilocks may still find the porridge too slow. SSDs turn out to be fast, but not as fast as you might hope in out of the box configurations. MySQL on SSD is fast for sequential reads and writes, but random reads are writes are relatively slow. Kevin speculates "Log structured filesystems can come into play here and seriously increase performance by turning all random writes into sequential writes" and that "Bigtable and append only databases would FLY on flash."
The upshot for me is we need a storage engine designed specifically for SSDs as they provide a very different design space from hard disks. Kevin has a lot of excellent details and observations on his site and he'll no doubt be coming up with a lot more.
Reader Comments (4)
I would like to see servers like these coming with the SSD options
http://www.supermicro.com/products/system/1U/
Apparently all of them are stuck with some HDD reqirements.
Too bad that this technology will see the light of the day when in might be too late!
Kevin,
Sorry that you had such a bad time with mysql 4.1.x and pbxt engine.
You did realize that pbxt engine was built for 5.1x and above?
Todd,
Help me.. The author is has a misunderastanding?
In his blog (the one you point us to), he says
The MySQL 4.1.x source snapshot would dump core on install.
(he knows that this engines works on 5.1+?)
Compiling PBXT with MySQL 5.1.x is a comedy of errors.
the last I remember, it was a simple follow the INSTALL file
To clarify your issues above:
"You did realize that pbxt engine was built for 5.1x and above?"
No... current sources are designed for 5.1.x. There is a snapshot of PBXT integrated into the 4.1.x sources.
http://www.primebase.com/xt/download/mysql-4.1.16-pbxt-0.9.6.tar.gz
Only later versions require 5.1.x. That said. I had to install PBXT with 5.1.x to get it to work.
"the last I remember, it was a simple follow the INSTALL file"
Did you actually try it? The INSTALL file is wrong. Read my comment on my blog post.
It requires additional configure options to work.