Scaling an AWS infrastructure - Tools and Patterns

This is a guest post by Frédéric Faure (architect at Ysance), you can follow him on twitter.

How do you scale an AWS (Amazon Web Services) infrastructure? This article will give you a detailed reply in two parts: the tools you can use to make the most of Amazon’s dynamic approach, and the architectural model you should adopt for a scalable infrastructure.

I base my report on my experience gained in several AWS production projects in casual gaming (Facebook), e-commerce infrastructures and within the mainstream GIS (Geographic Information System). It’s true that my experience in gaming (IsCool, The Game) is currently the most representative in terms of scalability, due to the number of users (over 800 thousand DAU – daily active users – at peak usage and over 20 million page views every day), however my experiences in e-commerce and GIS (currently underway) provide a different view of scalability, taking into account the various problems of availability and data management. I will therefore attempt to provide a detailed overview of the factors to take into account in order to optimise the dynamic nature of an infrastructure constructed in a Cloud Computing environment, and in this case, in the AWS environment.

Click to read more ...

Hot Scalability Links for Aug 13, 2010

• Ezra Zygmuntowicz in a heart warming account of his 4 Years at Engine Yard, has concluded in his experience that: the true future of cloud computing for developers is to not think about servers at all. It is now time to focus on the Application and new levels of abstraction that allow folks to use the computing resources in easier and easier ways. 
• Tweets of Gold:
  • bryanlatten: Nothing like a million caching layers to screw up an already complicated deployment. Thankfully, there is beer.
  • jkalucki: Twitter isn't down, you are just using the wrong access methods...
  • andyedinborough: I don't mean to hate, but why would I give up performance and scalability for a dynamic language? Honestly, I don't get it.
  • AsitSinha: It's amazing.... to see the absence of an understanding of how capability plays a role in scalability.

Click to read more ...

Strategy: Terminate SSL Connections in Hardware and Reduce Server Count by 40%

This is an interesting tidbit from near the end of the Packet Pushers podcast Show 15 – Saving the Web With Dinky Putt Putt Firewalls. The conversation was about how SSL connections need to terminate before they can be processed by a WAF (Web Application Firewall), which inspects HTTP for security problems like SQL injection and cross-site scripting exploits. Much was made that if programmers did their job better these appliances wouldn't be necessary, but I digress.

To terminate SSL most shops run SSL connections into Intel based Linux boxes running Apache. This setup is convenient for developers, but it's not optimized for SSL, so it's slow and costly. Much of the capacity of these servers are unnecessarily consumed processing SSL.

Click to read more ...

Think of Latency as a Pseudo-permanent Network Partition

The title of this post is a quote from Ilya Grigorik's post Weak Consistency and CAP Implications. Besides the article being excellent, I thought this idea had something to add to the great NoSQL versus RDBMS debate, where Mike Stonebraker makes the argument that network partitions are rare so designing eventually consistent systems for such rare occurrence is not worth losing ACID semantics over. Even if network partitions are rare, latency between datacenters is not rare, so the game is still on.

The rare-partition argument seems to flow from a centralized-distributed view of systems. Such systems are scale-out in that they grow by adding distributed nodes, but the nodes generally do not cross datacenter boundaries. The assumption is the network is fast enough that distributed operations are roughly homogenous between nodes.

Click to read more ...

Sponsored Post: Okta, EzRez, VoltDB, Digg, Cloud Sigma, Applications Manager, Site24x7

Who's Hiring?

• Okta is hiring! Okta provides a ground-breaking cloud adoption and management solution and they are looking for people in many different areas.
• ezRez is a B2B SaaS provider looking to hire a Senior Software Engineer and Software Engineer

Cool Products and Services

• Cloud Sigma. Instantly scalable European cloud servers. 
• ManageEngine Applications Manager.  ManageEngine provides Enterprise IT Management suite of products. 
• Site24x7. Easy, fast and effective web server monitoring, server monitoring and website monitoring service.

---

Okta is Hiring for Many Positions in Engineering, Marketing, Sales, and Customer Success

We're building a key service for the cloud, in the cloud, by people who know the cloud. Our team is composed of people who were central to building services likes Salesforce.com and SuccessFactors, systems which process millions of transactions in the cloud every day. We know what problems people are having because we experienced them ourselves and saw our customers and colleagues cry for help. We're changing the way people interact with technology, starting with a very fundamental element: identity.

We've got exciting, hard problems to solve and we want you to help us. We learned a lot while creating the largest on-demand enterprise companies, and we're putting that knowledge to good use as we build the next generation of corporate IT. At Okta we understand what Internet-scale innovation requires, which is why we've started fresh, with no legacy code or old code lines to maintain. It's a fast-paced, agile environment – just like the Internet – and we need the best and the brightest to help us change the world.

For more information see Okta's Careers page. 

---

VoltDB Field/Community Engineer

VoltDB is attracting more and more users every day. If you have a strong technical background in SQL and Linux, are experienced with production database deployments, and have a passion for customers and community, you could be just the person we are looking for.  Are you excited about the prospect of working with users to develop and deploy VoltDB applications, and about helping users participate in the thriving VoltDB community? If so, read on at their job page.

---

Get Your High Scalability Fix at Digg 

Interested in working on cutting-edge high-scale infrastructure at Digg? We're making a big investment in scaling and have committed to the NoSQL (Not only SQL) path with Cassandra. We're using other open-source infrastructure to help us scale including Hadoop, RabbitMQ, Zookeeper, Thrift, HDFS and Lucene. We're rewriting Digg from the ground up and we need amazing developers to join our world-class team. If you think you are up for the challenge, or you know someone who might be, take a look at our jobs page for more information.

---

CloudSigma

• Instantly Scalable European Cloud Servers. Create virtual servers in the cloud that are fully scalable and adaptive. Control your servers via our web console or API. CloudSigma gives more power and control over your server infrastructure.
• Keep control, increase scalability. Subscribe for capacity or pay as you go; with CloudSigma we give you the power and control you need.
• Competitive Innovative Pricing. Discover transparent pricing and a flexible billing model. Purchase what you need when you need it without resource bundling. We let you purchase CPU, RAM, Storage and bandwidth independently. Create your perfect combination that’s right for you. 
• 14-day Free Trial. Try our cloud computing products free. 

More information at CloudSigma.

---

ManageEngine Applications Manager

ManageEngine provides Enterprise IT Management suite of products. ManageEngine Applications Manager helps SaaS companies monitor their production applications and helps keep costs low.
There is out-of-the-box support for monitoring application servers, database servers, servers and web servers from a single web console. In addition to support for IBM Applications, Oracle Apps and Microsoft applications, there is deep support for Open Source Applications like JBoss, Memcached, LAMP stack etc.  Pricing starts at $795 for monitoring 25 servers or applications. Learn more about the Application Performance Monitoring tool.

---

Site24x7

Site24x7.com (from ZOHO) is a Website and Web Application Monitoring service. It helps you ensure your shopping carts and other web transactions work. It also helps you monitor the performance of your websites from a global point of presence. You can Sign Up for a Free Trial. The Professional Edition starts at $1 / Month. Learn more about the Website Monitoring Service.

---

ezRez Senior Software Engineer

You will be part of a fun, fast-paced and highly collaborative engineering team leveraging agile methodologies to deliver new functionality in incremental iterations. You will get exposed to cutting-edge technologies including an open source stack, dependency injection (DI) frameworks, ORM, XML web services, distributed grid-based caching and the latest UI technologies. We're also currently using best-of-breed extreme programming (XP) techniques including test driven development, pair programming, continuous refactoring and continuous integration.

Requirements

• Familiarity with agile software development and a desire to champion it among the team.
• Ability to work independently, multitask and manage time effectively.
• Experience with test-driven development techniques and/or well-disciplined to write unit tests that assert something useful.
• Excellent communication skills (verbal, written, wiki, and white-boarding).
• 5+ years experience with Java (or other OO languages like C++, Smalltalk, Ruby, Python) with deep understanding of object-oriented design.
• Passion for technology outside the workplace with an interest in the latest open source framework/libraries/tools including Spring, Hibernate, concurrency, memcached/key-value repositories, Freemarker, Tomcat, subversion, maven, and Ruby on Rails.

Nice to Have

• Familiarity with the travel industry or a high-transaction ecommerce web-site.
• Experience with a hosted, multi-tenant application environment.
• We're global, so a familiarity with internationalization and configurable displays is a plus.
• Experience with OWASP secure coding guidelines.

How to Apply

Please email your resume (as an attachment) to employment@ezrez.com with the subject "Sr. Software Engineer" for immediate consideration. You may also contact our recruiter via Skype at JanetBourland. Please note that at this time we are not considering candidates that will require employer sponsorship to work in the United States. No calls from recruiters/agencies please. ezRez Software is an equal opportunity employer.

---

ezRez Software Engineer

You will be part of a fun, fast-paced and highly collaborative engineering team leveraging agile

methodologies to deliver new functionality in incremental iterations. You will get exposed to

cutting-edge technologies including an open source stack, dependency injection (DI)

frameworks, ORM, XML web services, distributed grid-based caching and the latest UI

technologies. We're also currently using best-of-breed extreme programming (XP) techniques

including test driven development, pair programming, continuous refactoring and continuous

integration.

Requirements

• Ability to work independently, multitask and manage time effectively.
• Some exposure to agile software development or a desire to practice it.
• 1-3 years experience with Java (or other OO languages like C++, Smalltalk, Ruby, Python) with a firm grasp of object-oriented design
• Experience with test-driven development techniques and/or well-disciplined to write unit tests that assert something useful.
• Excellent communication skills (verbal, written, wiki, and white-boarding).
• Passion for technology outside the workplace with an interest in the latest open source framework/libraries/tools including Spring, Hibernate, concurrency, memcached/key-value repositories, Freemarker, Tomcat, subversion, maven, and Ruby on Rails.

Nice to Have

• Familiarity with the travel industry or a high-transaction ecommerce web-site.
• Experience with a hosted, multi-tenant application environment.
• We're global, so a familiarity with internationalization and configurable displays is a plus.
• Experience with OWASP secure coding guidelines.

How to Apply

Please email your resume (as an attachment) to employment@ezrez.com with the subject "Software Engineer" for immediate consideration. You may also contact our recruiter via Skype at JanetBourland. Please note that at this time we are not considering candidates that will require employer sponsorship to work in the United States. No calls from recruiters/agencies please. ezRez Software is an equal opportunity employer.

---

If you are interested in a sponsored post for an event, job, or product, please take a look at the advertising section.

Hot Scalability Links for Aug 6, 2010

• Twitter Sees Its 20 Billionth Tweet writes  Marshall Kirkpatrick of ReadWriteWeb.
• Startups die for not having customers, so STOP thinking about how to scale. Alessandro Orsi says focusing on the architecture and scaling possibilities of your app for millions of users is just plain dumb...concentrate on marketing...concentrate on user experience. Alessandro is perfectly correct, but this isn't the year the 2000 when the default architecture that is easy is also not scalable and when sites were built from scratch one painful user at a time.  Today neither is tue. In the era of social networks, where Facebook has 500 million users, successful applications can and often do spike to millions of users seemingly overnight. And you have to have some architecture. With today's tool-chains you don't have to choose easy and non-scalable. There are other options. Of course, it's all pointless without customers and that is what you need to worry about, but it's a false choice in this era to think that's all you have to worry about.

Click to read more ...

Dremel: Interactive Analysis of Web-Scale Datasets - Data as a Programming Paradigm

If Google was a boxer then MapReduce would be a probing right hand that sets up the massive left hook that is Dremel, Google's—scalable (thousands of CPUs, petabytes of data, trillions of rows), SQL based, columnar, interactive (results returned in seconds), ad-hoc—analytics system. If Google was a magician then MapReduce would be the shiny thing that distracts the mind while the trick goes unnoticed. I say that because even though Dremel has been around internally at Google since 2006, we have not heard a whisper about it. All we've heard about is MapReduce, clones of which have inspired entire new industries. Tricky.

Dremel, according to Brian Bershad, Director of Engineering at Google, is targeted at solving BigData class problems:

While we all know that systems are huge and will get even huger, the implications of this size on programmability, manageability, power, etc. is hard to comprehend. Alfred noted that the Internet is predicted to be carrying a zetta-byte (1021 bytes) per year in just a few years. And growth in the number of processing elements per chip may give rise to warehouse computers of having 1010 or more processing elements. To use systems at this scale, we need new solutions for storage and computation.

Click to read more ...

7 Scaling Strategies Facebook Used to Grow to 500 Million Users

Robert Johnson, a director of engineering at Facebook, celebrated Facebook's monumental achievement of reaching 500 million users by sharing the scaling principles that helped reach that milestone. In case you weren't suitably impressed by the 500 million user number, Robert ratchets up the numbers game with these impressive figures:

• 1 million users per engineer
• 500 million active users
• 100 billion hits per day
• 50 billion photos
• 2 trillion objects cached, with hundreds of millions of requests per second
• 130TB of logs every day

How did Facebook get to this point?

Click to read more ...

Basho Lives up to their Name With Consistent Smashing

For some Friday Fun nerd style, I thought this demonstration from Basho on the difference between single master, sharding, and consistent smashing was really clever. I love the use of safety glasses! And it's harder to crash a server with a hammer than you might think...

Recommended reading:

• http://labs.google.com/papers/bigtable.html
• http://research.yahoo.com/project/212

Hot Scalability Links for July 30, 2010

• Jeremy Zawodny, while performing data alchemy in the dungeons of Craigslist, stored 1,250,000,000 Key/Value Pairs in Redis on a 32GB Machine.
• Data sorting world record: 1 terabyte, 1 minute. The system has 52 computer nodes, each node is a commodity server with two quad-core processors, 24 gigabytes (GB) memory and sixteen 500 GB disks. It's not just hardware though, they also built a software that utilized all their CPU and RAM.
• Tweets of Gold:
  • wm: I am really getting the sense that none of you yokels waxing profound about scalability actually has anything factual to say
  • joestump: I think you can do things to *mitigate* pain points up front. You don't need to over-engineer, but it's not hard to look forward.
  • danielcrenna: I love it when I check in debug code accidentally and it turns into a three day hunt for a major scalability problem
  • joestump: Your post also makes me think of another phrase I say often: Scaling == Specialization. Bigger scale = More specialization.

Click to read more ...