Saturday
Sep012007
On-Demand Infinitely Scalable Database Seed the Amazon EC2 Cloud
Saturday, September 1, 2007 at 3:21PM
Amazon's EC2 sounds good, but how do you make use of all that throbbing CPU power? A few companies are stepping up to fill the how-to gap. Elastra provides unlimited on-demand creation of MySQL and PostgresSQL instances for $.50/server/hour. They contend their clusters perform "nearly" as well as a local database deployed using local storage.
RightScale says they "enable you to run your entire web business on Amazon Web Services with reliability, scalability and performance – and pushbutton control of complex system administration tasks." This includes web servers, DNS, and MySQL services. Prices start at $500 a month.
Later I'll write more about these and other related services like 3tera, but these services are the canary in the coal mine, the face of change, the bellwether of the new data center. How we build scalable web sites is about to change.
RightScale says they "enable you to run your entire web business on Amazon Web Services with reliability, scalability and performance – and pushbutton control of complex system administration tasks." This includes web servers, DNS, and MySQL services. Prices start at $500 a month.
Later I'll write more about these and other related services like 3tera, but these services are the canary in the coal mine, the face of change, the bellwether of the new data center. How we build scalable web sites is about to change.
Reader Comments (16)
50 cents an hour = 360/month + bandwidth at 18cents/GB + S3 Storage, you will be looking at ~$500/month + bandwidth per node that's 1.7Ghz single core system. Given how cheap Intel's quad core Xeons are now a days, you can have a dual quad 8 core (2.4Ghz each core) system with 8-16GB of ram and 4-8 drive RAID10 storage for around $3000. Colocation of that server in a datacenter will provide you about 10 times the performance of a virtual SQL node. Provided that bandwidth costs are similar, then within one month, you can recoup all the cost of the server itself.
What I am saying is that while the solution to scale mysql or postgreSQL is very easy, it's prohibitively expensive to scale this way. Not only are you permanently married to EC2/S3, For one instance of the mySQL(single core only), you can colocate an entire rack in the datacenter
if you do decide on scaling this way, choosing mySQL over postgreSQL because postgreSQL is better scaling vertically(4-8 cores), while mySQL's replication is better at horizontal scaling. MySQL also works faster on single core systems which is what ec2 instances are and get killed at 4core +
the pricing needs to be 15 cents - 25 cents/hour, giving you 100-200/month, which is comparable to colocation..(still a slower single core verses if you were to build a system, a dual core/quad core now is minimum Xeon X3220)
The pricing for these services will be interesting. At first blush they seem high to me too. But let's consider some ameliorating issues:
1. Lease vs Buy. You can get in the game with lower capital costs and incremental scaling costs. This is an important issues for a lot of startups. Less so when you have deep pockets.
2. Managed servers. These servers are managed. If you don't have your own admin team this is important. Building a SAN for storage is also very expensive and technically daunting, so if you need an nice storage network leasing one makes a lot of sense.
3. Inherent scalability model and path. These services give you path of how to scale and the ability to scale on demand through their model. That's attractive if you aren't quite sure what to do. And how many people can set up and manage a clustered database system? That they make it is easy is nice.
4. Some costs are fixed. Bandwidth within Amazonplex is free, so you are only paying for bandwidth you would have had to pay for anyway.
5. More features. In particular, 3tera has a presence in multiple data centers, so you can set up a geographically distributed system much more easily.
For the technologically sophisticated with time and motivation, doing it yourself will always seem the most cost effective and powerful approach. But there are a lot of people for whom this isn't possible.
Though a $360/month list price may seem comparable to a hosted solution, you are getting a lot more. Besides not having to buy hardware, software, bandwidth, or storage (we give 1 Terabyte of S3 storage free), you get the ability to really run your solution on demand.
Imagine allocating hardware to set up a 10 node cluster, setting up databases on it, having users or analytical programs hook up to it for a month of heavy traffic or heavy reporting. Imagine having to expand those clusters to 20 nodes or contract them to 3 because you are doing so well, or you no longer need the processing power. Now, think about being able to do these things at a minute's notice.
Imagine having 3 versions of your complete Data Warehouse for testing, running all three at the same time for, say, three days, picking out the best design, tools, analytical engines and taking the other two down by clicking an icon while archiving the other two in case you want to try them later.
The point with ELASTRA is not just that it's the first solution that actually uses S3 as a disk drive and not a tape drive, but that is also helps the application vendor create, scale, manage, and deploy their solution as their enterprise (hopefully) grows.
I know hosting people will scream that they can also add servers easily, but for the screamers: your competition is Amazon, not ELASTRA. We don't have anything against you if you can offer our platform the same capabilities and the same prices as Amazon can and so effectively does.
"Managed servers. These servers are managed. If you don't have your own admin team this is important. Building a SAN for storage is also very expensive and technically daunting, so if you need an nice storage network leasing one makes a lot of sense."
It is interesting...someone should benchmark how many IOPS Amazon's S3 SAN gives you. Compared to say a 8x36GB 15KRPM SAS array(1200 dollars)
Pricing in the young utility computing space will be in flux for some time as the market decides which models work and which don't. If Elastra has solved the problem of running a database on EC2, I suspect they'll do well and the pricing will work itself out.
I don't know if Elastra has posted benchmark results, but we did post benchmarks for our Virtual Private Datacenter at http://www.3tera.com/benchmark_results.html.
One point I often see overlooked when discussing/evaluating utility computing solutions are SLA's (Service Level Agreements) and what recourses exist when the unthinkable does happen. Realizing that this approach to infrastructure is still in its infancy, almost all of the major players in this space have had some issues with downtime and performance that are worth mentioning; and in most cases the SLA's behind the services don't have enough teeth to protect your investment. After all, if this is the foundation upon which you are running your entire business, there needs to be enforceable safeguards to ensure the availability & performance of your applications. That type of assurance is difficult to acheive in large shared systems because the risk of a single problem affecting most or all clients is too great. Until a utility computing provider finds a way to offer a stringent SLA and can deliver on it, this approach will always remain on the sidelines of truly enterprise deployments.
I completely agree that it is not just cost and benchmarks. I remember working at Oracle in the late 80s when benchmarks were all the rage. Oracle was just releasing Version 6 which was really THE FIRST of its RDBMS products that was taken seriously in production, transactional environments. Still what were we doing for the next 6-7 years??? -- decision support systems and things like CRM while business ran on MVS and the AS/400. My point is that you are right, and also that it will take a while before "cloud computing" is taken seriously by serious IT departments. It will be a while before my ATM transactions are stored in an "elastic database." But, one has to start somewhere. Things like salesforce automation (note success of salesforce.com), human resources (taleo) and lots of other on-demand vendors just getting started. What's needed is backing from some serious vendors (which we are now getting) and a new pricing model that truly measures utility instead of CPU's, licenses, and users.
It's a new wave of computing and it needs time to swell.
Amazon's EC2 sounds good, but how do you make use of all that throbbing CPU power? A few companies are stepping up to fill the how-to gap. Elastra provides unlimited on-demand creation of MySQL and PostgresSQL instances for $.50/server/hour. They contend their clusters perform "nearly" as well as a local database deployed using local storage.
RightScale says they "enable you to run your entire web business on Amazon Web Services with reliability, scalability and performance – and pushbutton control of complex system administration tasks." This includes web servers, DNS, and MySQL services. Prices start at $500 a month.
Later I'll write more about these and other related services like 3tera, but these services are the canary in the coal mine, the face of change, the bellwether of the new data center. How we build scalable web sites is about to change.
__________________
Submited by : http://www.todoendietas.com/dietas/dietafibras.html">Dietas
> instances for $.50/server/hour.
That's $360 a month + $200/TB/month for S3. The Amazon charge is $.10/hour for EC2 and $150/TB for S3. Do you think that's a good deal?
Some of the posters here keep comparing EC2 and S3 with traditional boxes filling up data centers. Whether you host your own servers or are paying someone else to do it at their datacenter, your deployment is strictly tied to that hardware. What happens when you need to scale short term (say, following a major press event or promotion)? Well, if you own the boxes, you could run out and buy another one, build it, install your WFE/DB or whatever else it's going to run and hope that that new box you just shelled out for is going to cut it. Alternatively you could call your hosting provider and ask them for another dedicated box and kindly request that they hurry, but in most cases you're looking at about 24-48 hours for new resources. Regardless, your scalability and resources are still tied to physical boxes in single geographic locations.
With EC2 you're looking at 5-10 minutes to boot up and instance loaded with your AMI and (well, using RightScale's tools) load balanced into your full deployment cluster. What if your master DB fails? Promote your slave and bring up a new slave (takes about 10-15 minutes). What about resources for grid computing tasks (transcoding media, batch processing forms, etc)? Fire up 10 new instances to take care of the load and shut them back down in an hour (that just cost you a dollar). If you have to test a new deployment, as a poster pointed out, you could bring up 2, 5, 10, even 1000 new servers just to test it, and then shut them down in a day. Moreover these virtual resources are not geographically locked in and a total datacenter failure would not take down your deployment. Utility computing is changing the way resources are delivered.
To those that are comparing it price wise to traditional hosting, you're certainly not putting any value on your time and management resources. How many people are you paying a year's salary to run and maintain your servers? How much are you paying them when you need more? What is your time worth as a system administrator? Just make sure that hardware costs aren't your only consideration, if they are, then you're probably not quite ready for AWS. Every solution requires a need in the first place.
Good points Matt. I think SoftLayer quoted me turn on times of a few hours, but the point still stands. For top tier properties what is missing from the mix is a geographical datacenter model. What you get with BigTable and Dynamo is multi-datacenter operation with nearest datacenter routing. You can open datacenters near your customers in Europe, Asia, and South America, which is important for latency and redundancy. What happens "automatically" is replication, syncing, caching, routing, upgrades, etc. To date you don't get all that with cloud services.
I agree Todd. Amazon is currently lacking the groundwork to be in some of those continents, but your key phrase there was "to date".
Amazon is primarily concerned with the bare metal infrastructure and the trend seems to be that they are intentionally designing their products to be feature poor but highly configurable. Take their new SDB offering; it's very basic and lacks a lot of the features you get with MySQL, Postgres, Oracle, etc. However, for applications that are optimized for that simplicity, it's going to scream!
Besides, the need for a management layer (offering replication, syncing, caching, routing and so on) on top of the AWS offerings is why people use the RightScale platform to manage their deployment. We've already created those tools. (Didn't mean to advertise, but it's certainly worth mentioning in context!) Upgrades in terms of the hardware backing the service are kind of irrelevant since the resources are virtual and on demand.
What we're seeing is that, in it's current form, startups and larger companies interested in pilot projects on AWS are flocking to this infrastructure early to test its limits and prepare themselves for the future of the utility computing industry.
Has anyone reported using 'utilities' like AWS and as a result, letting go of sysadmin or DBA personnel ?
Are there any real-world reports of organizations using EC2 with MySQL, and at what query rates ? While the hyperbole of 'on-demand, inifitely-scalable' sounds great, I'd like to hear from the largest deployments, and whether or not they think that they'll outgrow AWS.
So then there has been a lot of changes with amazon with respect to the databases they are handling with mysql.
-----
http://underwaterseaplants.awardspace.com">sea plants
http://underwaterseaplants.awardspace.com/seagrapes.htm">sea grapes...http://underwaterseaplants.awardspace.com/seaweed.htm">seaweed