Start Here
Friday, July 6, 2007 at 6:55AM This page is here to help you get started using High Scalability. Here are a few useful topics to get you going...
- Why does the High Scalability site exist?
- Good things to read.
- Participate by adding your own links to interesting sites and articles.
- Participate by signing up for the RSS feed.
- Consider the many benefits of registering as a user.
- How do I get notification of content and comment changes?
- Contact High Scalability.
- About.
Why does the High Scalability site exist?
To help you build successful scalable websites.
This site tries to bring together all the lore, art, science, practice, and experience of building scalable websites into one place so you can learn how to build your website with confidence.
When it becomes clear you must grow your website or die, most people have no idea where to start. It's not a skill you learn in school or pick up from a magazine article on a plane flight home. No, building scalable systems is a body of knowledge slowly built up over time from hard won experience and many failed battles. Hopefully this site will move you further and faster along the learning curve of success.
Makers of popular web sites eventually run into this all important question: How do I scale? Every builder of successful web sites must answer and that question and put their answers into practice.
You might wonder:
- How do I handle being digged or slashdotted?
- What can I accomplish on my budget?
- How do I add more and more users?
- What software should I use? LAMP, WAMP, or .Net?
- Should I use managed or unmanaged systems? Dedicated, co-located, VPS, hosting, or something else?
- Which machine and OS should I use?
- How do I recover from a disaster?
- How do I measure and improve performance?
- Where do I get people to help me?
- Which data center should I use?
- Which ISP should I use?
- How can I structure my software to scale?
- How do we setup caching?
- What should my database schema look like?
- Which database should I use?
- Which language and framework should I use?
- How do I ensure my data is always available and never lost?
- How do I monitor all my software and machines?
- How do I train my programmers to build this type of software?
- How do I failover my web servers, databases, etc?
- How do I expand to multiple geographical locations?
- How should I handle session data?
- How do I handle support and upgrades and feature rollouts?
You probably have 1000s of questions like these. Where do you find the answers? The answers are out there. How to build a scalable website is not a secret, the information is just spread out. And it's still more art than science. Every problem is different. Your site may have specific requirements that make it just different enough that you could use some advice.
And that's what this site is all about. Bringing like-minded people together to help each learn everything we can about creating the best websites we can.
Good things to read.
If you are interested in this site then you probably want to build your own monster website. There's no better way to learn than learning from the best Real Life Architectures out there. Real Life Architectures is a continuing series of posts on how real successful websites like eBay, Flickr, MySpace,LiveJournal, and Amazon build their websites.
Learn from those who have already done it and add your own personal twist to make it your own.
Participate by adding your own links to interesting sites and articles.
One of the incredible free to the user rewards for registering as a user of High Scalability is that you can post articles to the front page. The amount of materials on High Scalability topics is vast and ever evolving, so if people share what they find that will help everyone keep up on what's new.
To post your own articles all you have to do is:
- Register as a user.
- Login.
- Select the post new entry link at the top of the page.
- Fill out the form and click on submit. And you're done!
Participate by signing up for the RSS feed.
If you would like to participate in this web site by reading RSS postings then just paste the following URL into your favorite RSS reader: http://feeds.feedburner.com/HighScalability. For the comment feed subscribe to http://highscalability.squarespace.com/blog/rss-comments.xml.
Consider the many benefits of registering as a user.
OK, to be honest, there aren't that many benefits of registering as a user. We hate sites that make you register before you can do anything useful. We've made it so you can do most everything interesting without registering. But if you do register you can post an article and subscribe to article changes via email. That's about it. Hopefully we'll have some nice door prizes later.
How do I get notification of content and comment changes?
- RSS is probably the easiest way. Please see the RSS section a little earlier on the page.
- Registered users can login and select Subscribe to Page Updates in the upper right hand corner of the page.
Posting Rules
I really want to let people post whatever they find interesting, but this is the Internet. So, please:
- Do not post announcements for new product versions, classes, etc. Soon the site would become nothing but announcements, people would stop reading, and that would be bad. Please send me your announcement and I can fold it into to my weekly hot links post.
About
Some people have asked who I am. Good question. I am still working on that :-) My name, however, is Todd Hoff and my personal website is at http://toddhoff.com. I have a lot of experience in large scale distributed systems and a long standing interest in the subject. I finally decided since I'm reading this stuff all the time I might as well start a site about it!
I hope you find this site useful in your day-to-day work in the trenches.
Mini FAQ
- Why do some of your pages look horrible? During the move from Drupal to Squarespace, I had to export the content and try to automatically reformat it to look presentable on Squarespace. Regrettably that process didn't always work. I made a lot of hand edits to fix whatever I could, but there's a lot of content on this site and I missed a lot. So I'm trying to slowly improve it. If there's a page you think needs attention please let me know.
Reader Comments (25)
And tons of information! Good Job!!
I have never seen such a web site which is so much focussed on web site architecture. I am enlightened by the bountiful information it contains!
reply
Thanks a zillion. There is a lot of information I really want to know. Your site answers a lot of my questions.
Hi !
Certainly your site is again the *real* source on the architectural front.
I just have felt that it can be improved by adding the architectural diagrams[not exactly but kind of pics that visualize the architectures you are describing].
But,hats off for your collections and contribution as a single source for this information!
Just Curious
Hi Todd!
Thanks a ton for coming up with a site that has exclusive and extensive information on subjects I always wanted to study about. The architecture of YouTube was good and so are other real life applications, keep up the good work!
Cheers!
As soon as I saw that Drupal was being used and that there were 110+ guests online, I knew that this must be an authoritative site for information. Great work!
Very nice site! Congratulations. http://evandro.net/">Evandro
For beginners to experts, this site is so much focussed on HA stuffs that I am overwhelmed to see it! Never seen a site like this (applies only to me). howtoforge.com blended with this site would be a wonderful and almost complete resource!! Kudos to TMH and no words to appreciate his effort!!!
Zenny
Your website was a very valuable resource for information that helped us building our website. Today our website gets a few million hits per day and I am sure that without information that we got from your website, especially caching related issues, it wouldn't be able to handle that amount of traffic using only two low-budget servers and open source software. Thank you!
cool site thnaks...
This is one of the best site, I came across when searching for PoF architecture and ending up architecture quiet a few popular site.
Invaluable papers and articles. Thanks for sharing.
Good Web site for sharing these knowledge. Thanks.
This website by Todd has provided really valuable information on scalability over the year. Real life architectures of some of the existing websites has been specially helpful.
By the way, this website has also been mentioned in the list of programming and software development blogs here-> http://www.talkora.com/technology/List-of-programming-and-software-development-blogs_105 (look for entry #27 in the list.)
Almost 3 years of experienced Software guy here. But I'm fairly new to all things web but I've always been amazed and intrigued by how huge sites operate and everything. Although I don't understand (yet) half of the information in these articles, I love reading them. Hopefully one day, I'll be able to make my own contributions as well.
Friendly tip: You really need to remove the
text-transform: capitalize;CSS on H2 elements. It Makes It Harder To Read Because It Distorts The Word Shapes That Our Brains Use To Read. It Also Results In Capitalization Of Words That Shouldn't Be Capitalized, Like "The", "A", "To", "And", "By", "Of", "For". It is also disconnected from your style of writing H2's as sentences.Nice site. Can I publish tech related story and news?
This site really really has lot of valuable information for developing high scalable applications. A one stop place for all the HA info. Thanks a lot Todd. Keep it going. Awesome work.
Thanks for having this fantastic platform with bountiful information purely focused on scalability - Thanks
Bookstore link doesn't work.
Thanks for letting me know. The book store is gone the way of all things digital I'm afraid.
Just started reading and have a feeling that this site is the one that I have been looking for :-). Great introduction, looking forward to learning and sharing.
Thanks a lot.
A lot of good content, but the site itself is terrible. Have you ever thought about creating a Medium publication instead of this?
Just wanted to let you know that the link towards "Real Life Architecture Examples" points to a subdomain under `squarespace.com`, which is probably not what you want. Thank you for the content.