Tuesday
Sep302008
Scalability Worst Practices
Tuesday, September 30, 2008 at 4:40AM
Brian Zimmer, architect at travel startup Yapta, highlights some worst practices jeopardizing the growth and scalability of a system:
* The Golden Hammer. Forcing a particular technology to work in ways it was not intended is sometimes counter-productive. Using a database to store key-value pairs is one example. Another example is using threads to program for concurrency.
* Resource Abuse. Manage the availability of shared resources because when they fail, by definition, their failure is experienced pervasively rather than in isolation. For example, connection management to the database through a thread pool.
* Big Ball of Mud. Failure to manage dependencies inhibits agility and scalability.
* Everything or Something. In both code and application dependency management, the worst practice is not understanding the relationships and formulating a model to facilitate their management. Failure to enforce diligent control is a contributing scalability inhibiter.
* Forgetting to check the time. To properly scale a system it is imperative to manage the time alloted for requests to be handled.
* Hero Pattern. One popular solution to the operation issue is a Hero who can and often will manage the bulk of the operational needs. For a large system of many components this approach does not scale, yet it is one of the most frequently-deployed solutions.
* Not automating. A system too dependent on human intervention, frequently the result of having a Hero, is dangerously exposed to issues of reproducibility and hit-by-a-bus syndrome.
* Monitoring. Monitoring, like testing, is often one of the first items sacrificed when time is tight.
* The Golden Hammer. Forcing a particular technology to work in ways it was not intended is sometimes counter-productive. Using a database to store key-value pairs is one example. Another example is using threads to program for concurrency.
* Resource Abuse. Manage the availability of shared resources because when they fail, by definition, their failure is experienced pervasively rather than in isolation. For example, connection management to the database through a thread pool.
* Big Ball of Mud. Failure to manage dependencies inhibits agility and scalability.
* Everything or Something. In both code and application dependency management, the worst practice is not understanding the relationships and formulating a model to facilitate their management. Failure to enforce diligent control is a contributing scalability inhibiter.
* Forgetting to check the time. To properly scale a system it is imperative to manage the time alloted for requests to be handled.
* Hero Pattern. One popular solution to the operation issue is a Hero who can and often will manage the bulk of the operational needs. For a large system of many components this approach does not scale, yet it is one of the most frequently-deployed solutions.
* Not automating. A system too dependent on human intervention, frequently the result of having a Hero, is dangerously exposed to issues of reproducibility and hit-by-a-bus syndrome.
* Monitoring. Monitoring, like testing, is often one of the first items sacrificed when time is tight.
Reader Comments (6)
I've found monitoring gets developed better if it (alongside tests) are the way the developers test the system. If the test runs include the monitor info, and that is how you have to work out what went wrong, you add good diagnostics.
"The Golden Hammer. Forcing a particular technology to work in ways it was not intended is sometimes counter-productive. Using a database to store key-value pairs is one example."
So what is the best-practice way of persistently storing key-value pairs? Don't say memcached as that isn't persistent.
> So what is the best-practice way of persistently storing key-value pairs? Don't say memcached as that isn't persistent.
berkely db's are a good candidate for key/value pair lookup:
http://en.wikipedia.org/wiki/Berkeley_DB
Take a look at memcachedb too. Next week I'll have something on how Digg is using memcachedb for their some high write tasks.
The growing IT sector of Bangalore is no longer news. So much so that the number of http://www.fortunehotels.in/HotelResort/main.aspx?ID=jjzMpzSjmet8b1CqrZapPg==&PType=6FrpaFUg4tzWXwacpHoKmg==">business hotels in Bangalore have tremendously increased after the entry and subsequent flourish of the IT sector.
Stating seemingly disingenious things without justification is just provocative and devalues the entire article:
"Another example is using threads to program for concurrency."
This isn't a contribution to the community, its soap-box aloofness.