Scaling Memcached: 500,000+ Operations/Second with a Single-Socket UltraSPARC T2

A software-based distributed caching system such as memcached is an important piece of today's largest Internet sites that support millions of concurrent users and deliver user-friendly response times. The distributed nature of memcached design transforms 1000s of servers into one large caching pool with gigabytes of memory per node. This blog entry explores single-instance memcached scalability for a few usage patterns. Table below shows out-of-the-box (no custom OS rewrites or networking tuning required) performance with 10G networking hardware and one single-socket UltraSPARC T2-based server with 8 cores and 8 threads per core (64 threads on a chip)... Object Size / Ops/Sec / Bandwidth 100 bytes / 530,000 / 1.2 Gb/s 2048 bytes / 370,000 / 6.9 Gb/s 4096 bytes / 255,000 / 9.2 Gb/s Check out the link for more details!

Click to read more ...

Scalability Perspectives #6: Lew Tucker – Virtual Data Centers in the Open Cloud

Scalability Perspectives is a series of posts that highlights the ideas that will shape the next decade of IT architecture. Each post is dedicated to a thought leader of the information age and his vision of the future. Be warned though – the journey into the minds and perspectives of these people requires an open mind.

Lew Tucker

Lew Tucker is the Vice President and CTO of Sun Microsystems’ Cloud Computing initiative. Lew’s career has been focused on scalable computing and web development. Lew worked at Sun Microsystems through the 1990’s. In 2002, Lew joined Salesforce.com and led the design and implementation of App Exchange. After Salesforce.com, Lew was CTO at Radar Networks, where he focused on the scalable design and build out of its semantic web service.

The Sun Cloud API

Sun has recently announced its open RESTful API for creating and managing cloud resources, including compute, storage, and networking components. Lew and his team is busy implementing Sun's cloud offering. His background and experience from the creation of Salesforce App Exchange has helped the team to create a simple but flexible set of APIs. Lew envisions open, interoperable clouds based on community standards such as the Cloud API.

Your Own Virtual Data Center in the Cloud

Lew Tucker has demonstrated the use of the Sun Cloud by building a Virtual Data Center built of virtual servers, switches and storage. He commented on the announcement in this interview. Check out the information sources below to understand the perspective of Lew Tucker on Cloud Computing and learn more about the relationship of the Sun and the Clouds. The TechCrunch roundtable is especially interesting.

Information Sources

• Lew Tucker on CrunchBase, Freebase, Project Kenai, Twine
• InformationWeek: Sun's Innovative Drag, Drop, & Deploy Virtual Datacenter Designer
• eWeek interview: Sun Cloud Group CTO Talks Sun Cloud
• The Open Sun Cloud APIs
• Cloud Computing at Sun
• TechCrunch: Cloud Computing Roundtable
• Overcast Show #5: Dec 11, 2008 - with Lew Tucker, Sun Microsystems

Click to read more ...

Sun to Announce Open Cloud APIs at CommunityOne

One of the key items Sun will be talking about in today's cloud computing announcement (at 9AM EST/6AM PST) will be Sun's opening of the APIs that we'll use for the Sun Cloud. We're making these available so that those who are interested will be able to review and comment on these APIs. Continuing our commitment to openness, we're making these APIs available via the Creative Commons Version 3.0 license. ...

Click to read more ...

Cisco and Sun to Compete for Unified Computing?

A recent InfoWorld article claims that "With Cisco expected to enter the blade market and Sun expected to offer networking equipment, things could get interesting awfully fast." How does this effect your infrastructure strategy and decisions? Would you consider to build scalable web applications on the Cisco Unified Computing System? Or would you consider to build a router out of a server with the use of OpenSolaris and Project Crossbow as the article suggests? Will any of these initiatives change the way we build scalable web infrastructure or are these just attempts to sale these systems? What do you think?

Click to read more ...

Sun Acquires Q-layer in Cloud Computing Play

Datacenterknowledge.com: In an effort to boost its refocused cloud computing initiative, Sun Microsystems (JAVA) has acquired Q-layer, a Belgian provider that automates the deployment of both public and private clouds. Sun says Q-layer’s technology will help users instantly provision servers, storage, bandwidth and applications. Do you have experience with Q-layers technology like its Virtual Private DataCenter and NephOS?

Click to read more ...

Sun FireTM X4540 Server as Backup Server for Zmanda's Amanda Enterprise 2.6 Software

Sun FireTM X4540 Server as Backup Server for Zmanda's Amanda Enterprise 2.6 Software by Thomas Hanvey (Sun Microsystems) and Dmitri Joukovski and Ken Crandall (Zmanda) September, 2008 Explosive data growth, combined with demanding requirements for data availability, has placed a tremendous burden on IT operations staff at businesses of all sizes. Yet, many organizations do not have the staff or budget to purchase and manage complex and expensive backup and recovery software products. The Sun FireTM X4540 server can deliver massive storage capacity and remarkable throughput so it is well-suited as a nearline storage platform for backup and restore applications. Combining the power of the SolarisTM 10 Operating System with the data integrity and simplified administration of ZFS, the Sun Fire X4540 server can be an ideal candidate for streamlining and improving backup and restore operations. Amanda Enterprise Edition from Zmanda was designed to address these challenges, providing a backup and recovery solution that combines fast installation, simplified management, enterprise-class functionality, and low-cost subscription fees. As an open source product, Amanda Enterprise Edition uses only standard formats and tools, effectively freeing you from being locked into a vendor to recover your archived data. This guide discusses how to quickly configure the Sun Fire X4540 server as a backup server for Amanda Enterprise Edition software.

Click to read more ...

eBay Architecture

Update 2: EBay's Randy Shoup spills the secrets of how to service hundreds of millions of users and over two billion page views a day in Scalability Best Practices: Lessons from eBay on InfoQ. The practices: Partition by Function, Split Horizontally, Avoid Distributed Transactions, Decouple Functions Asynchronously, Move Processing To Asynchronous Flows, Virtualize At All Levels, Cache Appropriately. Update: eBay Serves 5 Billion API Calls Each Month. Aren't we seeing more and more traffic driven by mashups composed on top of open APIs? APIs are no longer a bolt on, they are your application. Architecturally that argues for implementing your own application around the same APIs developers and users employ. Who hasn't wondered how eBay does their business? As one of the largest most loaded websites in the world, it can't be easy. And the subtitle of the presentation hints at how creating such a monster system requires true engineering: Striking a balance between site stability, feature velocity, performance, and cost. You may not be able to emulate how eBay scales their system, but the issues and possible solutions are worth learning from. Site: http://ebay.com

Information Sources

The eBay Architecture - Striking a balance between site stability, feature velocity, performance, and cost.

Podcast: eBay’s Transactions on a Massive Scale

Dan Pritchett on Architecture at eBay interview by InfoQ

Platform

Java

Oracle

WebSphere, servlets

Horizontal Scaling

Sharding

Mix of Windows and Unix

What's Inside?

This information was adapted from Johannes Ernst's Blog

The Stats

On an average day, it runs through 26 billion SQL queries and keeps tabs on 100 million items available for purchase.

212 million registered users, 1 billion photos

1 billion page views a day, 105 million listings, 2 petabytes of data, 3 billion API calls a month

Something like a factor of 35 in page views, e-mails sent, bandwidth from June 1999 to Q3/2006.

99.94% availability, measured as "all parts of site functional to everybody" vs. at least one part of a site not functional to some users somewhere

The database is virtualized and spans 600 production instances residing in more than 100 server clusters.

15,000 application servers, all J2EE. About 100 groups of functionality aka "apps". Notion of a "pool": "all the machines that deal with selling"...

The Architecture

Everything is planned with the question "what if load increases by 10x". Scaling only horizontal, not vertical: many parallel boxes.

Architectures is strictly divided into layers: data tier, application tier, search, operations,

Leverages MSXML framework for presentation layer (even in Java)

Oracle databases, WebSphere Java (still 1.3.1)

Split databases by primary access path, modulo on a key.

Every database has at least 3 on-line databases. Distributed over 8 data centers

Some database copies run 15 min behind, 4 hours behind

Databases are segmented by function: user, item account, feedback, transaction, over 70 in all.

No stored procedures are used. There are some very simple triggers.

Move cpu-intensive work moved out of the database layer to applications applications layer: referential integrity, joins, sorting done in the application layer! Reasoning: app servers are cheap, databases are the bottleneck.

No client-side transactions. no distributed transactions

J2EE: use servlets, JDBC, connection pools (with rewrite). Not much else.

No state information in application tier. Transient state maintained in cookie or scratch database.

App servers do not talk to each other -- strict layering of architecture

Search, in 2002: 9 hours to update the index running on largest Sun box available -- not keeping up.

Average item on site changes its search data 5 times before it is sold (e.g. price), so real-time search results are extremely important.

"Voyager": real-time feeder infrastructure built by eBay.. Uses reliable multicast from primary database to search nodes, in-memory search index, horizontal segmentation, N slices, load-balances over M instances, cache queries.

Lessons Learned

Scale Out, Not Up – Horizontal scaling at every tier. – Functional decomposition.

Prefer Asynchronous Integration – Minimize availability coupling. – Improve scaling options.

Virtualize Components – Reduce physical dependencies. – Improve deployment flexibility.

Design for Failure – Automated failure detection and notification. – “Limp mode” operation of business features.

Move work out of the database into the applications because the database is the bottleneck. Ebay does this in the extreme. We see it in other architecture using caching and the file system, but eBay even does a lot of traditional database operations in applications (like joins).

Use what you like and toss what you don't need. Ebay didn't feel compelled to use full blown J2EE stack. They liked Java and Servlets so that's all they used. You don't have to buy into any framework completely. Just use what works for you.

Don't be afraid to build solutions that meet and evolve with your needs. Every off the shelf solution will fail you at some point. You have to go the rest of the way on your own.

Operational controls become a larger and larger part of scalability as you grow. How do you upgrade, configure, and monitor thousands of machines will running a live system?

Architectures evolve. You need to be able to change, refine, and develop your new system while keeping your existing site running. That's the primary challenge of any growing website.

It's a mistake to worry too much about scalability from the start. Don't suffer from paralysis by analysis and worrying about traffic that may never come.

It's also a mistake not to worry about scalability at all. You need to develop an organization capable of dealing with architecture evolution. Understand you are never done. Your system will always evolve and change. Build those expectations and capabilities into your business from the start. Don't let people and organizations be why your site fails. Many people will think the system should be perfect from the start. It doesn't work that way. A good system is developed overtime in response to real issues and concerns. Expect change and adapt to change.

Click to read more ...

Sun to Acquire MySQL

So what are we announcing today? That in addition to acquiring MySQL, Sun will be unveiling new global support offerings into the MySQL marketplace. We'll be investing in both the community, and the marketplace - to accelerate the industry's phase change away from proprietary technology to the new world of open web platforms. Read more on Jonathan Schwartz's Blog What do you think about this?

Click to read more ...