The Mullet Cloud Selection Pattern

In a recent thread on Hacker News one of the commenters mentioned that they use Digital Ocean for personal stuff, but use AWS for business.

This DO for personal and AWS for business split has become popular enough that we can now give it a name: the Mullet Cloud Selection Pattern - business on the front and party on the back.

Providers like DO are cheap and the lightweight composable container model has an aesthetic appeal to developers. Even though it seems like much of the VM infrastructure has to be reinvented for containers, the industry often follows the lead of developer preference.

The mullet is dead. Long live the mullet! Developers are ever restless, always eager to move onto something new. 

The “Four Hamiltons” Framework for Mitigating Faults in the Cloud: Avoid it, Mask it, Bound it, Fix it Fast

This is a guest post by Patrick Eaton, Software Engineer and Distributed Systems Architect at Stackdriver.

Stackdriver provides intelligent monitoring-as-a-service for cloud hosted applications.  Behind this easy-to-use service is a large distributed system for collecting and storing metrics and events, monitoring and alerting on them, analyzing them, and serving up all the results in a web UI.  Because we ourselves run in the cloud (mostly on AWS), we spend a lot of time thinking about how to deal with faults in the cloud.  We have developed a framework for thinking about fault mitigation for large, cloud-hosted systems.  We endearingly call this framework the “Four Hamiltons” because it is inspired by an article from James Hamilton, the Vice President and Distinguished Engineer at Amazon Web Services.

The article that led to this framework is called “The Power Failure Seen Around the World”.  Hamilton analyzes the causes of the power outage that affected Super Bowl XLVII in early 2013.  In the article, Hamilton writes:

As when looking at any system faults, the tools we have to mitigate the impact are: 1) avoid the fault entirely, 2) protect against the fault with redundancy, 3) minimize the impact of the fault through small fault zones, and 4) minimize the impact through fast recovery.

The mitigation options are roughly ordered by increasing impact to the customer.  In this article, we will refer to these strategies, in order, as “Avoid it”, “Mask it”, “Bound it”, and “Fix it fast”...

Click to read more ...

10 Things You Should Know About AWS

Authored by Chris Fregly:  Former Netflix Streaming Platform Engineer, AWS Certified Solution Architect and Purveyor of fluxcapacitor.com.

Ahead of the upcoming 2nd annual re:Invent conference, inspired by Simone Brunozzi’s recent presentation at an AWS Meetup in San Francisco, and collected from a few of my recent Fluxcapacitor.com consulting engagements, I’ve compiled a list of 10 useful time and clock-tick saving tips about AWS.

1) Query AWS resource metadata

 

Can’t remember the EBS-Optimized IO throughput of your c1.xlarge cluster?  How about the size limit of an S3 object on a single PUT?  awsnow.info is the answer to all of your AWS-resource metadata questions.  Interested in integrating awsnow.info with your application?  You’re in luck.  There’s now a REST API, as well!

Note:  These are default soft limits and will vary by account.

2) Tame your S3 buckets

 

Delete an entire S3 bucket with a single CLI command:  

aws s3 rb s3://<bucket-name> --force

Recursively copy a local directory to S3:

aws s3 cp <local-dir-name> s3://<bucket-name> --region <region-name> --recursive

3) Understand AWS cross-region dependencies

Click to read more ...

5 Rockin' Tips for Scaling PHP to 30,000 Concurrent Users Per Server

Jonathan Block, CTO at RockThePost.com, a crowdfunding company, has written a nice set of tips for smaller sites on how to scale a service on EC2 using a small two person development team. 

Their service has a typical small scale structure:

• PHP's Zend Framework 2
• Two m1.medium for web servers
• ELB to split the load
• master/slave MySQL database
• Siege for load testing

The very sensible tips that can handle 30,000 concurrent users per web server: 

Click to read more ...

A Simple 6 Step Transition Guide for Moving Away from X to AWS

If you just want to visit Rome and not go full on Cloud Native like Netflix, then Soundslice's Adrian Holovaty in Why I left Heroku, and notes on my new AWS setup provides a simple guide for helping make your first trip a good one.

First, let's dispose of why Soundslice left Heroku. The essence is because of various issues "Heroku lost my trust." YMMV, but once a fact, what do you do?

After a consultation with Scott VanDenPlas, former director of dev ops for the Obama reelection tech team, they came up a simple transition guide that I think is quite good and generally useful (full details in the original post): 

Click to read more ...

Is Provisioned IOPS Better? Yes, it Delivers More Consistent and Higher Performance IO

Amazon created a whole new class of service with their Provisioned IOPS for RDS, EBS, and DynamoDB. The idea is simple. If you want more performance, you turn a dial up. If you want less, you turn a dial down. A beautifully simple model. You pay for the performance you want, which is different than their previous cloud model, where performance varied, but you paid only for what you used. 

The question: Do these higher priced services really work better?

Rodrigo Campos put this question to the test (only for EBS) by running a benchmark he describes in IOMelt Provisioned IOPS EBS Benchmark Results - December 2012.

The result? Yes, AWS Provisioned IOPS Volumes Really Deliver More Consistent and Higher Performance IO:

Click to read more ...

Peecho Architecture - scalability on a shoestring

This is a guest post by Marcel Panse and Sander Nagtegaal from Peecho.

Although architecture descriptions are an interesting read, the problems that start-ups face are hardly ever addressed. We would like to change that, so here is our architecture story.

Introducing a start-up

The Amsterdam-based company Peecho offers print-as-a-service. Our embeddable print button allows you to sell your digital content as professionally printed products, like photo books, magazines or canvases - straight from your own website. There is an API, too.

Printcloud is the system that powers the print button. It exists in the cloud only, growing when needed and becoming smaller if it can. The system takes in print orders, magically transforms tough data into print-ready files and routes the orders to the production facility that is closest to the intended recipient.

To preserve the environment, Peecho's philosophy is to facilitate global ordering, but to aim for local production only.

Expensive stuff does not scale

Click to read more ...

Automation on AWS with Ruby and Puppet

This is a guest post by Frédéric Faure (architect at Ysance), you can follow him on twitter.

Urbandive is an immersive view service launched by the French YellowPages which allows you to travel in cities in France thanks to a 360° view. Urbandive focuses on providing high definition pictures and accurate professional and social content. One of the biggest jobs was to enable a fast scalable architecture, because it was really difficult to forecast the traffic load at production time. Traffic load may be influenced if the service receives attention from users as a result of advertising.

Below you will find a summary of the goals we achieve by using a Ruby scheduler built on top of Puppet on AWS to create a complete infrastructure.

Click to read more ...

Paper: Netflix’s Transition to High-Availability Storage Systems

In an audacious move for such an established property, Netflix is moving their website out of the comfort of their own datacenter and into the wilds of the Amazon cloud. This paper by Netflix's Siddharth “Sid” Anand, Netflix’s Transition to High-Availability Storage Systems, gives a detailed look at this transition and does a deep dive on SimpleDB best practices, focussing especially on techniques useful to those who are making the move from a RDBMS.

Sid is going to give a talk at QCon based on this paper and he would appreciate your feedback. So if you have any comments or thoughts please comment here or email Sid at r39132@hotmail.com or Twitter at @r39132 Here's the introduction from the paper:

Click to read more ...

Scaling an AWS infrastructure - Tools and Patterns

This is a guest post by Frédéric Faure (architect at Ysance), you can follow him on twitter.

How do you scale an AWS (Amazon Web Services) infrastructure? This article will give you a detailed reply in two parts: the tools you can use to make the most of Amazon’s dynamic approach, and the architectural model you should adopt for a scalable infrastructure.

I base my report on my experience gained in several AWS production projects in casual gaming (Facebook), e-commerce infrastructures and within the mainstream GIS (Geographic Information System). It’s true that my experience in gaming (IsCool, The Game) is currently the most representative in terms of scalability, due to the number of users (over 800 thousand DAU – daily active users – at peak usage and over 20 million page views every day), however my experiences in e-commerce and GIS (currently underway) provide a different view of scalability, taking into account the various problems of availability and data management. I will therefore attempt to provide a detailed overview of the factors to take into account in order to optimise the dynamic nature of an infrastructure constructed in a Cloud Computing environment, and in this case, in the AWS environment.

Click to read more ...