Design Preparations for Scaling

Hi there,

what do you think is crucial in the code designing of a scalable site? How does one prepare for webfarms and clusters (e.g. in PHP)?

Thanks,
Stephan

Comments

[Todd Hoff]

Take a look at the different profiles on this site is the best answer, but not the answer you probably want to hear :-) There are lots of different approaches and lessons to learn. The big lessons IMHO are using a shared nothing architecture, partition, cache, use what you know, start small, build in monitoring and logging, profile, remove bottlenecks, use a service architecture, make use of all the good open source software, take a look at a CDN, plan on how you'll grow your storage, and be open to doing things differently than you've done before. Using PHP or another language isn't the really the major issue.

[Stephan]

Thanks for the comment!
I really love your site, was really interesting to the BIG architectures.

I like your advise, shows me where to keep an eye on. However, what do you exactly mean by "shared nothing architecture" and "service architecture"?

Thanks,
Stephan

[Todd Hoff]

In a shared nothing system the state for your application is reconstituted from the database (or other sources) for each operation. You don't keep live objects around between requests. So when a http request comes into the web server you go to the database to get the state for the request, process the request, and write state back to the database. Your processes are short lived and the state lives in the database. The advantage is scalability and simplicity. You can simply add more web servers, for example, and you can scale as far as your database will take you. Since you don't have to lock data against shared access, it's also much simpler to program. You may find "The Case for Shared Nothing" interesting reading (http://db.cs.berkeley.edu/papers/hpts85-nothing.pdf).

In a service architecture you internally access features of your system via a RPC like interface. It could be REST or whatever. So instead of apps using a DAO to go against the database directly you contact a service to do the work for you. What's behind the service is then independent and can be developed independently and made scalable on its own terms, apart from the database. Take a look at http://highscalability.com/amazon-architecture">Amazon Architecture for more details.