In Scalability issues for dummies Alex Barrera talks movingly about the challenges he faces trying to scale his startup inkzee as the lone developer. Inkzee is an online news reader that automatically groups similar topics. This is a cool problem and is one you know right away is going to have some killer scalability problems as the number of feeds and the number of users increase. And these problems lead to the point of the post, to explain here what are scalability problems and how deep the repercussions are for a small company, which Alex does admirably.
Some takeaways:
Sites are composed of a frontend and backend. The backend isn't visible to users, but it does all the work and this is where the scalability problems show up.
As more and more users use a site it becomes slow because more users reveal bottlenecks in the system that weren't visible before.
There can be many many reasons for these bottlenecks. They are often very hard to find because the backend systems are very complex and have a lot of complex interactions.
It takes a while to find the bottlenecks and create fixes for them. You are never quite sure if the fixes will really work. Many of these problems are unique to the problem space so pre-canned solutions aren't always available. And because you don't want to destroy your production servers it takes a while to put fixes into the system. This means your release cycle is slow which means progress on your site is slow.
The process of redesign sucks up all your resources so progress on the site stalls almost completely, especially for a small development group. This stops growth as you can't handle new customers and your existing customers become disenchanted.
The process is unfortunately iterative. Solving one problem just puts you in the queue for the next problem. There's a reason it took Twitter a while to get on their feet.
Scalability problems aren’t something you can discard as being ONLY technical, it’s roots might be technical but its effects will shake the whole company.
I found Alex's commentary quite touching and familiar. As I imagine many of you do too. It's the modern equivalent of an explorer following a dream. Going alone into uncharted territory where the Dragons live and trying to survive when everything seems against you. For every great returning hero there are 10 who do not make it back. And that's hard to deal with.
Everyone will certainly have their ideas on how to "fix" the problem, as that's what engineers do. But it also doesn't hurt to use our Venus brain for a moment and simply recognize the toll this process can take. It can be dispiriting. The continual stream of problems and lack of positive feedback can wear you down after a while. To stick with it takes a bit of craziness in the heart.
Switching back to being Mars brained I might suggest:
Find a partner. There's always a Jerry to go along with Ben. A Martin to go along with Lewis. And Rambo is just a movie. Going it alone is hard hard hard. Partners can pick each other up and give each other a breather when necessary.
Use the Cloud. Whatever your opinions of the economics of the cloud, it makes testing of the type Alex was needing a breeze. Architect for the cloud and you can spin up a system in parallel, run a load generator, and never touch your production system at all. And do it all for cheap. This is one of the biggest wins for the cloud and would seem to solve a lot of problems.
Related Articles
Scalable Web Architectures and Application State by Marton Trencseni
Scalability for dummies by Royans Tharakan
Monday, July 20, 2009 at 2:48AM
Reader Comments (1)
Hey Todd! First of all I feel honored to have been featured here. Thanks a lot!
As you say, the problem I'm tackling is a lot of fun but a hell of a scalability nightmare hehe I'm currently migrating the old database to a schemaless db (Tokyo Tyrant) and it's being a funny ride so far.
About the cloud, the whole architecture is distributed, with multiple master-worker nodes and it's ready to migrate to the cloud, something I'll do as soon as I finish the db migration, but you're right, the cloud is a blessing for scaling things, specially on the beginning.
I wrote this more as a way of showing a lot of people what is it to be there and how it's not only technical, but goes all the way to the business layers of a company.
I'm also a consultant at the largest social network in Spain so I've seen real funky scalability problems there hehe so I guess I can say I'm learning a lot about it lately.
About your partnership suggestion, you're very right and I wished I could find someone to help me out. The biggest problem is that I'm based in Madrid, Spain. The amount of web sites here that have to deal with scalability problems is tiny, so it's really hard to find a partner here. Nevertheless I don't despair, I know I'll eventually find someone :) (Be it here on in the Bay Area where I plan to move in a close future heh)
Again, thanks a lot for the post and the suggestions!! I feel very honored! :D