Nothing shows how much software architectures have changed than the intelligent scheduling of computation over differently priced compute resources. This isn't just a false economy either. Vimeo saves up to 50% on their video transcoding bill by intelligently playing the spot, reserved, and on-demand markets. If you are ready for some advanced reindeer games then take a look at Vimeo EC2 transcoding where they explain their thinking. Even if you don't like their rules, it's the strategy that matters. This presentation was from 2011, so it would be interesting to see if the new reserved instance market has made a difference in their strategy.
Here's Vimeo's approach for minimizing costs using spot, reserved, and on-demand instances:
- Never bid more than threshold. It is currently set to 80% of on-demand price.
- Not more than 10 open spot requests at any time.
- Bid 10% more than the average price over last hour
- Buy reserve instance capacity to meet non-peak hour loads.
- Use spots for low priority and less time critical jobs
- Never kill spots. Let Amazon do it.
- Have more retries for jobs running on spots.
- Watch out for open spot requests. Add expiry to your requests.
- Long running jobs, bid higher or use on-demand
- Failover to on-demand when spot marked is saturated