A Metric A$$-Ton of Joe Stump: The Cloud is Cheaper than Bare Metal

Should you pay more in the cloud or pay less for bare metal in the datacenter? This is a crucial decision point facing startups today. Which way should you go? In this Webpulp.tv interview, Joe Stump, always a go-to guy when you need a metric ass-ton (a favorite expression of Joe’s) of good advice on cutting edge practices for the modern startup, laughs at conventional wisdom by saying the cloud is really not more expensive than bare metal.

The argument for a cheaper cloud has a three main points:

1. Raw Hardware Costs are Close. Buying hardware is cheaper than renting hardware from Amazon, but it’s close when you consider reserved instances. A reserved instance costs about $250 a year and a new server costs about $1500. Let’s say Amazon is 30% slower. That means for a cluster of 20 machines you would need a another 7 servers. This takes 10 minutes and 4 button clicks to provision on the cloud (the advantage of a scale-out architecture + automation + cloud aware tools). So 7 extra servers using reserved instances costs another $7-10K a year. High end people cost $120K+, which is two weeks of work. At the end of the day would you have some spend 6 months building a new datacenter or spend more time producing direct value? Given a projected server life of 2 years, the costs will be close.
2. Hidden Costs. There are lot of hidden costs when running a datacenter that people usually don’t consider in their calculations. People usually just consider raw server costs, but that’s just the start of what you need: 1) devops people to spec hardware, work contracts, etc. 2) network engineers to manage routers, etc. These are very expensive people that do an essential job, but they don’t add direct value to your product, so why not let Amazon employ them?
3. Opportunity Costs. The real disadvantage of buying servers is the opportunity cost. With a small team of say 3 people, if one person spends a day in the datacenter then you’ve lost 33% of your productivity because time is being spent in the muck instead of providing direct value to customers. The idea is to have all your big guns working on your product, adding value, not working on plumbing. Hardware has been commoditized. A small shop can’t have its talent spend all their time wrangling hardware and sure as the sky is blue, someone will end up spending most of their time on hardware issues.

I think Joe makes a pretty convincing case here, especially for greenfield applications with smaller teams, with tight finances, that are using a scale-out scalability strategy. The incremental savings of bare metal is dwarfed by other concerns. And by concentrating all their effort on producing value they might grow into the position where going bare metal becomes a real decision for them. For now, however, it’s the flexibility of the cloud that is allowing them to make massive inroads with relatively little expense.

37signals offers direct counter example. 37signals is moving away from Rackspace and into their own datacenter. Some forces acting on 37signals are:

• They have an existing architecture that isn't cloud based. 37signals is successful and well established, what they are doing is working, so there's no incentive to change everything to a different model.
• They have a scale-up scaling strategy. With a scale-up strategy you need big hardware and the best way to get big hardware is to buy and manage it yourself.
• They have money. With money in the bank CapEx isn't your mortal enemy.
• They have three full-time sysadmins. With dedicated people, product development isn't harmed.

It’s possible SimpleGeo, Joe's current startup, may grow into a 37signals in the future and have to decide if increased profit margins are worth going bare metal. Or it’s possible Amazon will keep innovating and bring costs down and increase performance so that they will never have to move. In either case the main point for me was Joe’s pro tip of considering opportunity costs, not just raw server costs in cloud cost calculations.

Related Articles

1. Why I’ll never own another server by Joe Stump.
2. Nuts & Bolts: New Datacenter! - 37signals moves into a new datacenter.
3. Stack Overflow Architecture. This article has more discussion on the value of scale-up.
4. Strategy: Rule Of 3 Admins To Save Your Sanity

Comments

[John Hugg]

A reserved instance costs about $250 a year and a new server costs about $1500. Let’s say Amazon is 30% slower.

1. My math says reserving the smallest ec2 instance for 3 years and running it continuously costs about $380 a year.
2. That instance is supposedly equal to a 1.0ghz opteron with 1.7gb of memory. Assuming that's true, any $1500 server is going to be several times faster, not a mere 30%.

This part of the argument reads as crazy talk to me. That said, there are many factors other than the cost of the machine, some of which are mentioned here. I'm not neceesarily disagreeing with the conclusion.

[engsmith]

I am not sure why people always try to convince others that what worked for their special case should be applied in general. We just went through the similar process to evaluating EC2:

- We use bad-ass MySQL boxes with only SSD drives to handle massive amount of writes. Each box costs around $8K (48GB memory). In EC2, the similar size box's price is at least 3x more expensive, plus its I/O is no comparison with SSDs. Scale-out is a good strategy, however, when you can, why not scale-up a little, it can save you tons of complexity.

- Recently somebody mentioned that in bad days in EC2, you can't hold a TCP connection longer than 10 minute. Imagine what this will do to your distributed systems which relies on TCP to communicate status.

- Good System Admins are always required regardless if you use EC2 or colo. Negotiating pricing is a matter of doing business we shouldn't be shy away from. Just because EC2 doesn't allow you to negotiate price doesn't mean they are not ripping you off.

[Steve Loughran]

Don't forget storage costs. Once you go over a few hundred TB, there's a lot to be said for JBOD storage. Over time, storage demands only increase, directly proportionally to your data retention rate. Over time, costs of SATA storage decreases, though in a production datacentre, you only see those cost reductions when you replace the 1TB SATA HDDs with 2TB HDDs for the big dataset, SSDs for the front end

Where clouds are good is where you don't know the #of machines in demand and need to vary them, and you don't measure your static dataset in petabytes.

[Cristian B]

Not everyone needs to invest large amounts of money upfront into hardware, as there are 12-month and 24-month leasing opportunities as well as renting dedicated servers. If you have a constant requirement the cloud will cost you a pretty penny. For higher-end stuff this is very easily visible, as you have a 34.4GB memory instance from AWS EC2 costing over 700$ per month when run 24/7, without using their reserved instance, whereas a dedicated server with 32GB of RAM and two quad core Xeons can be had for less than 80% (without a contract, probably less with a 12month contract) of that price, depending on requirements, and will come with bandwidth included, and you don't share I/O with anyone.

I don't really understand why one person would need to spend a whole day in a datacenter. These days you can get stuff ready to rack and set up IPMI and handle things off-site, only needing to go into the DC to change parts.

The only time the cloud is convenient is when you truly need to scale, and I don't mean you've purposely built your small web app to scale so you can have with EC2, but actually have thousands of users pouring into your website at peak hours, requiring you to create additional instances.

[Crazy]

To add to the other good comments already posted:

If you only use a server for 2 years you're doing it wrong. You've forced the math to work for your article through widly inaccurate estimations and guesses.

Public cloud computing is great for two things: getting started and ramping up. Reddit is a prime example of a company that needs to move off of the cloud because the site has reached critical mass and the cloud is no longer affordable.

[Carson McDonald]

A lot of people seem to be involved in a debate about this right now. There is renewed talk on Reddit about how much they pay a month in hosting fees for EC2 (also on Hacker News about the same).

I think if you watch the entire video Joe does back off from his initial statements some. The opportunity cost argument is probably the strongest of the three. Everyone situation is going to be different and I think one of the places where EC2 shines is to fill a need for on demand and short lived processing power. The arguments start coming apart when you talk about static infrastructure.

As has been said in the comments already the 30% less power isn't close. Take for instance a Dell PowerEdge R410 that can give you 8 2.4Ghz cores and 16G of memory for $2k. That is 16 compute units on EC2. That is either 16 or 8 times as much as what you get with a standard EC2 instance.