Product: Memcached

memcached is a high-performance, distributed memory object caching system, generic in nature, but intended for use in speeding up dynamic web applications by alleviating database load. Danga Interactive developed memcached to enhance the speed of LiveJournal.com, a site which was already doing 20 million+ dynamic page views per day for 1 million users with a bunch of webservers and a bunch of database servers. memcached dropped the database load to almost nothing, yielding faster page load times for users, better resource utilization, and faster access to the databases on a memcache miss.

Click to read more ...

Book: Scalable Internet Architectures

As a developer, you are aware of the increasing concern amongst developers and site architects that websites be able to handle the vast number of visitors that flood the Internet on a daily basis. Scalable Internet Architecture addresses these concerns by teaching you both good and bad design methodologies for building new sites and how to scale existing websites to robust, high-availability websites. Primarily example-based, the book discusses major topics in web architectural design, presenting existing solutions and how they work. Technology budget tight? This book will work for you, too, as it introduces new and innovative concepts to solving traditionally expensive problems without a large technology budget. Using open source and proprietary examples, you will be engaged in best practice design methodologies for building new sites, as well as appropriately scaling both growing and shrinking sites. Website development help has arrived in the form of Scalable Internet Architecture.

Click to read more ...

BerkeleyDB & other distributed high performance key/value databases

I currently use BerkeleyDB as an embedded database http://www.oracle.com/database/berkeley-db/ a decision which was initially brought on by learning that Google used BerkeleyDB for their universal sign-on feature. Lustre looks impressive, but their white paper shows speeds of 800 files created per second, as a good number. However, BerkeleyDB on my mac mini does 200,000 row creations per second, and can be used as a distributed file system. I'm having I/O scalability issues with BerkeleyDB on one machine, and about to implement their distributed replication feature (and go multi-machine), which in effect makes it work like a distributed file system, but with local access speeds. That's why I was looking at Lustre. The key feature difference between BerkeleyDB and Lustre is that BerkeleyDB has a complete copy of all the data on each computer, making it not a viable solution for massive sized database applications. However, if you have < 1TB (ie, one disk) of total possible data, it seems to me that a replicated local key/value database is the fastest solution. I haven't found much discussion of people using this kind of technology for highly scalabable web sites. Over the years, I've had extremely good performance results with dbm files, and have found that nothing beats local data, access through C APIs, and btree or hash table implementations. I have never tried replicated/redundant versions of this approach, and I'm curious if others have, and what your experience has been.

Click to read more ...

What is Mashery?

In the Amazon Services architecture article the podcast mentions Mashery. I went to their site at http://www.mashery.com/, but I can't quite figure out what it is. They want to:

Unleash and manage channels for your API responsibly with Mashery’s combination of security, usage, access management, tracking, metrics, commerce, performance optimization and developer community tools.

An example would help, because I am not getting it.

Click to read more ...

Product: Yslow to speed up your web pages

Update: Speed up Apache - how I went from F to A in YSlow. Good example of using YSlow to speed up a website with solid code examples. Every layer in the multi-layer cake that is your website contributes to how long a page takes to display. YSlow, from Yahoo, is a cool tool for discovering how the ingredients of your site's top layer contribute to performance. YSlow analyzes web pages and tells you why they're slow based on the rules for high performance web sites. YSlow is a Firefox add-on integrated with the popular Firebug web development tool. YSlow gives you: * Performance report card * HTTP/HTML summary * List of components in the page * Tools including JSLint

Click to read more ...

Product: SmarterStats

SmarterStats provides a solid architecture businesses and individual end users can use to track growth and forecast internet trends. * Track your website's growth and forecast internet trends * Features over 130 report items, plus Geographic Reporting * Log comparison saving 90% of your disk space * Email Reports available in Enterprise Edition * Enhanced data mining available in both editions

Click to read more ...

Product: Akamai

Akamai transparently mirrors content (usually media objects such as audio, graphics, animation, video) stored on customer servers. Though the domain name is the same, the IP address points to an Akamai server rather than the customer's server. In addition to image caching, Akamai provides services which accelerate dynamic and personalized content, J2EE-compliant applications, and streaming media.

Click to read more ...

Product: Photobucket

Photobucket's free account has a storage limit and a download bandwidth limit of 10 GB per month. There's no bandwidth limit on the $25 Pro account.

Click to read more ...

Product: Flickr

Flickr offers a free basic account with limited upload bandwidth and limited storage. Download bandwidth is unlimited. Upgrading to a paid Pro account for $25/year removes all upload and storage restrictions. Flickr's terms of use warn that "professional or corporate uses of Flickr are prohibited", and all external images require a link back to Flickr.

Click to read more ...

Product: ImageShack

ImageShack offers free, unlimited storage with a 100 MB per hour bandwidth limit for each image.

Click to read more ...