Hot Scalability Links for April 30, 2010

• I Want a New Data Store. Jeremy Zawodny of Craigslist wants a new database, one that can do what it should: perform alter table operations faster, has efficient queries when most of the data is on disk and not in RAM, and matches their data that now looks more document oriented than relational. A lot of people willing to help.
• Computer Science Unplugged. An extensive collection of free resources that teach principles of Computer Science such as binary numbers, algorithms and data compression through engaging games and puzzles that use cards, string, crayons and lots of running around. And it's free! Fascinating Interview with Tim Bell on teaching complex computing concepts, creating makers not just users, and how to change schools. From O'Reilly Radar. 
• Akamai’s Network Now Pushes Terabits of Data Every Second. Akamai handles 12 million requests per second, logs more than 500 billion requests for content per day, and sends 3.45 terabits per second of data.
• Google’s MapReduce Programming Model — Revisited.  We reverse-engineer the seminal papers on MapReduce and Sawzall, and we capture our findings as an executable specification.
• Facebook Flashcache. James Hamilton describes "a simple write back persistent block cache designed to accelerate reads and writes from slower rotational media by caching data in SSD."
• Pigz – parallel gzip OMG. John Allspaw plays with a single core and multicore version of gzip. With one core zipping a 418m log file took 12.4 seconds. On a 16 core machine with parrallel gzip it to took 1.6 seconds.
• Hadoop Meetup Videos. Videos on Using Hadoop to fight spam at Yahoo! Mail, Hive/HBase integration, Public Terabyte Dataset Project - Web crawling with Amazon's EMR.
• How TokuDB Fractal Tree Indexes Work by Bradley C. Kuszmaul.  Fractal Trees are functionally equivalent to B-trees, but run significantly faster.  Fractal Trees convert random I/O, which involves painfully slow disk seeks, into sequential I/O, which provides up to two orders of magnitude more performance.  
• Scaling writes in MySQL by Philip Tellis. After partitioning (12 partitions per day, 2 hours of data per partition) we were able to sustain an insert rate of around 8500 rows per second.
• Attempts at Analyzing 19 million documents using MongoDB map/reduce by Steve Eichert. We still feel that Mongo will be a great place to persist the output of all of our Map/Reduce steps, however, we don’t feel that it’s well suited to the type of analysis that we want to do.
• TR10, and a Bloom/BOOM FAQ. BOOM is the name of a research project based at Berkeley, which seeks to enable programmers to build Orders Of Magnitude bigger systems in O.O.M. less code. Our focus is on enabling developers to easily harness the power of many computers at once, e.g. in the setting of cloud computing.
• Storing log messages in Hadoop by Peter Dikant. Using hadooop to store 20 TB of data.
• Ceph: The Distributed File System Creature from the Object Lagoon by Jeffrey B. Layton.  Ceph is a distributed parallel file system promising scalability and performance, something that NFS lacks.
• Horizontal Scalability via Transient, Shardable, and Share-Nothing Resources by Adam Wiggins. Now is the time of horizontal scalability achieved by using resources that are transient, shardable and share nothing with other resources. He gives as example several applications and a language: memcached, CouchDB, Hadoop, Redis, Varnish, RabbitMQ, Erlang, detailing how each one applies those principles.

If you would like to advertise a product, job, or event, please contact us for more information.