Database Sharding for startups

The most important aspect of a scalable web architecture is data partitioning. Most components in a modern data center are completely stateless, meaning they just do batches of work that is handed to them, but don't store any data long-term. This is true of most web application servers, caches like memcached, and all of the network infrastructure that connects them. Data storage is becoming a specialized function, delegated most often to relational databases. This makes sense, because stateless servers are easiest to scale - you just keep adding more. Since they don't store anything, failures are easy to handle too - just take it out of rotation.

Stateful servers require more careful attention. If you are storing all of your data in a relational database, and the load on that database exceeds its capacity, there is no automatic solution that allows you to simply add more hardware and scale up. (One day, there will be, but that's for another post). In the meantime, most websites are building their own scalable clusters using sharding.

Read more on LessonLearned blog.

Click to read more ...

Robert Scoble's Rules for Successfully Scaling Startups

Robert Scoble in an often poignant FriendFeed thread commiserating PodTech's unfortunate end, shared what he learned about creating a successful startup. Here's a summary of a Robert's rules and why Machiavelli just may agree with them:

Have a story.

Have everyone on board with that story.

If anyone goes off of that story, make sure they get on board immediately or fire them.

Make sure people are judged by the revenues they bring in. Those that bring in revenues should get to run the place. People who don't bring in revenues should get fewer and fewer responsibilities, not more and more.

Work ONLY for a leader who will make the tough decisions.

Build a place where excellence is expected, allowed, and is enabled.

Fire idiots quickly.

If your engineering team can't give a media team good measurements, the entire company is in trouble. Only things that are measured ever get improved.

When your stars aren't listened to the company is in trouble.

Getting rid of the CEO, even if it's all his fault, won't help unless you replace him/her with someone who is visionary and who can fix the other problems. An excellent list that meshes with much of my experience, which is why I thought it worth sharing :-) My take-away from Robert's rules can be summarized in one word: focus. Focus is the often over looked glue binding groups together so they can achieve great things... When Robert says have "a story" that to me is because a story provides a sort of "decision box" giving a group its organizing principle for determining everything that comes later. Have a decision? Look at your story for guidance. Have a problem? Look at your story for guidance. Following your stories' guidance is another matter completely. Without a management strong enough act in accordance with the story, centripetal forces tear an organization apart. It takes a lot of will to keep all the forces contained inside the box. Which is why I think Robert demands a "focused" leadership. Machiavelli calls this idea "virtue." Princes must exhibit virtue if they are to keep their land. Machiavelli doesn't mean virtue in the modern sense of be good and eat your peas, but in the ancient sense of manliness (sorry ladies, this was long ago). Virtue shares the same root as virility. So to act virtuously is to be bold, to act, to take risks, be aggressive, and make the hard unpopular decisions. For Machiavelli that's the only way to reach your goals in accordance with how the world really works, not how it ought to work. Any Prince who acts otherwise will lose their realm. Firing people (yes, I've been fired) not contributing to your story is by Machiavelli's definition a virtuous act. It is a messy ugly business nobody likes doing. It requires admitting a mistake was made, a lot of paperwork, and looking like the bad guy. And that's why people are often not fired as Robert suggests. The easy way is to just ignore the problem, but that's not being virtuous. Addition by subtraction is such a powerful force precisely because it maintains group focus on excellence and purpose. It's a statement that the story really matters. Keeping people who aren't helping is a vampire on a group's energy. It slowly drains away all vivacity until only a pale corpse remains. Robert's rules may seem excessively ruthless and cruel to many. Decidedly unmodern. But in true Machiavellian fashion let's ask what is preferable: a strong secure long-lived state ruled by virtue or a state ruled according to how the world ought to work that is constantly at the mercy of every invader? If you are still hungry for more starter advice, Gordon Ramsay has some unintentionally delicious thoughts on developing software as well. Serve yourself at: Gordon Ramsay On Software/04.html#a225"> and . Gordon Ramsay's Lessons for Software Take Two. Kevin Burton share's seven deadly sins startups should avoid and makes an inspiring case how his company stronger and better able to compete by not taking VC funds. Really interesting. Diary of a Failed Startup says solve a problem, not a platform to solve a class of problems. Truer words were never spoken.

Click to read more ...

Save on a Load Balancer By Using Client Side Load Balancing

In Client Side Load Balancing for Web 2.0 Applications author Lei Zhu suggests a very interesting approach to load balancing: forget DNS round robbin, toss your expensive load balancer, and make your client do the load balancing for you. Your client maintains a list of possible servers and cycles through them. All the details are explained in the article, but it's an intriguing idea, especially for the budget conscious startup.

Click to read more ...

Some Real Financial Numbers for Your Startup

If you are a startup you may find useful Guy Kawasaki's post Financial Models for Underachievers: Two Years of the Real Numbers of a Startup. Part of any business plan are the projected guestimates. They are guestimates because everyone keeps these numbers hidden like a Swiss bank account. But not Redfin. They've bravely shared their initial cost projections, their actual numbers from real life, and the lessons they've learned from the discrepancy between the two... You can find their model estimates and actuals for Rent, Per Employee, Per Month (model: $250, actual: $336); Initial Per-Employee Equipment Cost; Monthly Benefits, Per-Employee; Annual Payroll Tax; Quarterly Bonus Payout, as a % of the Total Possible; Annual Payroll Increase for Existing Employees; All-Company Meeting Cost, Per-Meeting, Per-Employee; Annual Accounting Costs, and a few more. There is also a great lessons section: Focus on headcount; Plan slow, run fast; Run top-down sanity-checks; Forget economies of scale; Admit that revenues are a mystery; Build from building blocks; Take out "hope"; Flag your assumptions; Hit $100 million in revenues within five years; Keep market-share under 20%. I find $100 million in revenues a surprisingly high number. That's a lot of money. And the underestimate for meeting costs is pretty funny. It's always those damn meetings!

Click to read more ...