Monday
Feb252008
Architecture Template Advice Needed
Monday, February 25, 2008 at 1:34PM
Here's my template for describing the architecture of a system. The idea is to have people fill out this template and that then becomes the basis for a profile. This is how the Friends for Sale post was created and I think that turned out well. People always want more detail, but realistically you can only expect so much.
The template is definitely too long, but it's more just a series of questions to jog people's memories and then they can answer whatever they think is important.
What I want to ask is if you can think of any things to add/delete/change in the template? What do you want to know about the systems people are building? So if you have the time, please take a look and tell me what you think.
* What is the name of your system and where can we find out more about it?
* What is your system is for?
* Why did you decide to build this system?
* How is your project financed?
* What is your revenue model?
* How do you market your product?
* How long have you been working on it?
* How big is your system? Try to give a feel for how much work your system does.
* Number of unique visitors?
* Number of monthly page views?
* What is your in/out bandwidth usage?
* How many documents, do you serve? How many images? How much data?
* How fast are you growing?
* What is your ratio of free to paying users?
* What is your user churn?
* How many accounts have been active in the past month?
* What is the architecture of your system? Talk about how your system works in as much detail as you feel comfortable with.
* What particular design/architecture/implementation challenges does your system have?
* What did you do to meet these challenges?
* How does your system evolve to meet new scaling challenges?
* Do you use any particularly cool technologies are algorithms?
* What do you do that is unique and different that people could best learn from?
* What lessons have you learned?
* Why have you succeeded?
* What do you wish you would have done differently?
* What wouldn't you change?
* How much up front design should you do?
* How are you thinking of changing your architecture in the future?
* How many people are in your team?
* Where are they located?
* Who performs what roles?
* Do you have a particular management philosophy?
* If you have a distributed team how do you make that work?
* What skillets does your team possess?
* What is your development environment?
* What is your development process?
* Is there anything that you would do different or that you have found surprising?
* Which languages do you use to develop your system?
* How many servers do you have?
* How is functionality allocated to the servers?
* How are the servers provisioned?
* What operating systems do you use?
* Which web server do you use?
* Which database do you use?
* Do you use a reverse proxy?
* Do you collocate, use a grid service, use a hosting service, etc?
* What is your storage strategy? DAS/SAN/NAS/SCSI/SATA/etc/other?
* How much capacity do you have?
* How do you grow capacity?
* Do you use a storage service?
* Do you use storage virtualization?
* How do you handle session management?
* How is your database architected? Master/slave? Shard? Other?
* How do you handle load balancing?
* Which web framework/AJAX Library do you use?
* Which real-time messaging frame works do you use?
* Which distributed job management system do you use?
* How do you handle ad serving?
* Do you have a standard API to your website? If so, how do you implement it?
* If you use a dynamic language which instruction caching product to use?
* What is your object and content caching strategy?
* What is your client side caching strategy?
* Which third party services do you use to help build your system?
* How do check global availability and simulate end-user performance?
* How do you health check your server and networks?
* How you do graph network and server statistics and trends?
* How do you test your system?
* How you analyze performance?
* How do you handle security?
* Do you implement web analytics?
* Do you do A/B testing?
! How is your data center setup?
* How many data centers do you run in?
* How is your system deployed in data centers?
* Are your data centers active/active, active/passive?
* How do you handle syncing between data centers and fail over and load balancing?
* Which firewall product do you use?
* Which DNS service do you use?
* Which routers do you use?
* Which switches do you use?
* Which email system do you use?
* How do you handle spam?
* How do you handle virus checking of email and uploads?
* How do you backup and restore your system?
* How are software and hardware upgrades rolled out?
* How do you handle major changes in database schemas on upgrades?
* What is your fault tolerance and business continuity plan?
* Do you have a separate operations team managing your website?
* Do you use a content delivery network? If so, which one and what for?
* How much do you pay monthly for your setup?
* Who do you admire?
* Have you patterned your company/approach on someone else?
* Are there any questions you would add/remove/change in this list?
The template is definitely too long, but it's more just a series of questions to jog people's memories and then they can answer whatever they think is important.
What I want to ask is if you can think of any things to add/delete/change in the template? What do you want to know about the systems people are building? So if you have the time, please take a look and tell me what you think.
Getting to Know You
* What is the name of your system and where can we find out more about it?
* What is your system is for?
* Why did you decide to build this system?
* How is your project financed?
* What is your revenue model?
* How do you market your product?
* How long have you been working on it?
* How big is your system? Try to give a feel for how much work your system does.
* Number of unique visitors?
* Number of monthly page views?
* What is your in/out bandwidth usage?
* How many documents, do you serve? How many images? How much data?
* How fast are you growing?
* What is your ratio of free to paying users?
* What is your user churn?
* How many accounts have been active in the past month?
How is your system architected?
* What is the architecture of your system? Talk about how your system works in as much detail as you feel comfortable with.
* What particular design/architecture/implementation challenges does your system have?
* What did you do to meet these challenges?
* How does your system evolve to meet new scaling challenges?
* Do you use any particularly cool technologies are algorithms?
* What do you do that is unique and different that people could best learn from?
* What lessons have you learned?
* Why have you succeeded?
* What do you wish you would have done differently?
* What wouldn't you change?
* How much up front design should you do?
* How are you thinking of changing your architecture in the future?
How is your team setup?
* How many people are in your team?
* Where are they located?
* Who performs what roles?
* Do you have a particular management philosophy?
* If you have a distributed team how do you make that work?
* What skillets does your team possess?
* What is your development environment?
* What is your development process?
* Is there anything that you would do different or that you have found surprising?
What infrastructure do you use?
* Which languages do you use to develop your system?
* How many servers do you have?
* How is functionality allocated to the servers?
* How are the servers provisioned?
* What operating systems do you use?
* Which web server do you use?
* Which database do you use?
* Do you use a reverse proxy?
* Do you collocate, use a grid service, use a hosting service, etc?
* What is your storage strategy? DAS/SAN/NAS/SCSI/SATA/etc/other?
* How much capacity do you have?
* How do you grow capacity?
* Do you use a storage service?
* Do you use storage virtualization?
* How do you handle session management?
* How is your database architected? Master/slave? Shard? Other?
* How do you handle load balancing?
* Which web framework/AJAX Library do you use?
* Which real-time messaging frame works do you use?
* Which distributed job management system do you use?
* How do you handle ad serving?
* Do you have a standard API to your website? If so, how do you implement it?
* If you use a dynamic language which instruction caching product to use?
* What is your object and content caching strategy?
* What is your client side caching strategy?
* Which third party services do you use to help build your system?
How do you manage your system?
* How do check global availability and simulate end-user performance?
* How do you health check your server and networks?
* How you do graph network and server statistics and trends?
* How do you test your system?
* How you analyze performance?
* How do you handle security?
How do you handle customer support?
How do you decide what features to add/keep?
* Do you implement web analytics?
* Do you do A/B testing?
! How is your data center setup?
* How many data centers do you run in?
* How is your system deployed in data centers?
* Are your data centers active/active, active/passive?
* How do you handle syncing between data centers and fail over and load balancing?
* Which firewall product do you use?
* Which DNS service do you use?
* Which routers do you use?
* Which switches do you use?
* Which email system do you use?
* How do you handle spam?
* How do you handle virus checking of email and uploads?
* How do you backup and restore your system?
* How are software and hardware upgrades rolled out?
* How do you handle major changes in database schemas on upgrades?
* What is your fault tolerance and business continuity plan?
* Do you have a separate operations team managing your website?
* Do you use a content delivery network? If so, which one and what for?
* How much do you pay monthly for your setup?
Miscellaneous
* Who do you admire?
* Have you patterned your company/approach on someone else?
* Are there any questions you would add/remove/change in this list?
Reader Comments (5)
I just read the 'Friends for Sale' post, and there were a number of items I just didn't care about. generally the "why did you decide to go into business?" questions. They're vague, and boring to me.
What I really want to see is more intelligent question and answer about their architecture. Unfortunately thats hard to put into a form. :) If you're sticking to the template system, Please ask that the folks illustrate both their logical application work flow (Load Balancer -> Proxy Server -> App Server -> MemCache -> MySQLD API Cluster -> Mysql Storage Node, etc) And also a physical architecture map/description (App servers are connected via redundant nics to a pair of Cisco 3845's, Dell 2950 Database Servers are connected to a iscsi Left Hand networks san Drive). It would be great to have both a visual MAP of both of these topics, AND a description/walkthrough of each step.
I'm much more interested in the nuts and bolts of these sites than I am the "big picture".. I think we all get the big picture by now "put the data you care about in multiple places, make sure you can get to it quickly, and make all the hardware cheap". For me the interesting thing is in the details of how each company accomplishes that.
Thanks
> What I really want to see is more intelligent question and answer about their
> architecture. Unfortunately thats hard to put into a form.
Yes it is, which is why I kind of punted on that section :-) Your suggestions are good and I'll add them in. It's surprisingly hard to talk about architecture. Even when you are in it you'll usually find it's not very well described. Even Google's documents are just the tip of a very large and fog enshrouded iceberg of a massively complex system.
> For me the interesting thing is in the details of how each company accomplishes that
How do should we talk about architecture? Mostly it's just boxes and circles connected by arrows. Look at an architecture document and it doesn't look much different that an interaction diagram for a software system.
What other questions would help tease out what people are doing in a way that would help other people along their way?
This is a classic principles versus detail debate. Do we concentrate on principles and bet people will uniquely express those principles in their own way for their problem for their own circumstance? Or does one give a hyperdetailed list of exact steps that isn't adaptable to other situations. Obviously, we want the best of both worlds, but that means we usually get the cold or hot porridge, not the one that's just right.
I think the best way to tease the information out, is asking them to walk through the flow of their architecture, either visually or as a description (preferably both). I also find that it helps to look at your architecture from two perspectives. Every time I build out a site for my clients, I draw a physical connection diagram, and a logical application flow diagram. I think where network diagrams fail is when people try to show both application flow and hardware flow in the same diagram.
By drawing the diagram's it forces you to think about each step, and to elaborate on why it works the way it does. I think the real gem's of information will be in the tidbits you jog from their memory while they're drawing the diagram up, and trying to explain exactly HOW and WHY they do something.
The problem with all this of course is that its time consuming, and fairly sensitive information. I would not be able to convince any of my clients to lay their network diagaram (visually or textually) out for the world to see. Have you ever offered to anonymize the data for people? So instead of saying Friends for sale, you could just describe it as a popular social facebook application focused on getting people laid. I don't use facebook, but I'm guessing there are a lot of those app's on facebook :)
If you ask a hyperdetailed list of questions, I think you end up with a bunch of answers to questions, instead of a thoughtful explanation of their architecture. Anybody who has ever worked for a large company, or a banking institution should know this feeling well.. When your department is audited based on a large list of questions, you start to focus your efforts on the list of questions, instead of whats really the "right thing to do". The result is usually cold pooridge. I think your template borders on that right now, but may or may not have crossed into that territory yet. IMHO its close though.
Unfortunately I have more critiques on this subject than I have good solutions. But please don't take my critiques harshly, I really enjoy your site, and as with all things, if you give me an inch, I'll want a mile. You do an EXCELLENT job of compiling information about companies that is very hard or impossible to find elsewhere. Good job! :) And I also thank the companies willing to share their hard earned details with others for the "betterment" of the internet community. Bravo.
> By drawing the diagram's it forces you to think about each step, and to elaborat
Excellent idea. Like how on CSI when you follow a bullet from the barrel of a gun and watch as it bounces around in some poor red shirt's body :-) I added this in.
> I think your template borders on that right now
Certainly. I don't really expect every question to be answered, but many questions just serve as memory joggers. And we do disagree about the soft issues. I like reading about more than just the tech details. Each project is a journey and I enjoy a bit of travelogue.
> So instead of saying Friends for sale, you could just describe it as a popular social facebook application focused on getting people laid
Not sure if this would succeed because people appreciate the recognition for their work. My hope is much like open source software, open source ideas make us all better. Getting those ideas and lessons out there multiplies everyones success.
> But please don't take my critiques harshly
Not at all. I wanted help! And I greatly appreciate your suggestions and conversation.
Hi,
If somebody can help me design the physical network diagram for the below configuration:.
I want to host 5 appilcation servers in high available (clustering environment) and database shouls also be in cluster envioronment . there will be 21 servers out of which 11 will be Linux and 8 will be windows. the applications which i will be using are running on the following platform and database as mentioned below:-
1. application 1 - Platform - RHEL 4 and database - mysql
2. app2 - ASP.net and mysql server (windows platform)
3. App3 - PHP/Windows flash/python/plone and mysql database
4. App4 - .NEt and MS SQL server
5. App5 - Html and windows
There will be one exchange server and we are using SAN storage .