Pinterest Architecture Update - 18 Million Visitors, 10x Growth,12 Employees, 410 TB of Data
Monday, May 21, 2012 at 9:15AM 
There has been an update on Pinterest: Pinterest growth driven by Amazon cloud scalability since our last post: A Short on the Pinterest Stack for Handling 3+ Million Users.
With Pinterest we see a story very similar to that of Instagram. Huge growth, lots of users, lots of data, with remarkably few employees, all on the cloud.
While it's true that both Pinterest and Instagram are not making great advances in science and technology, that is more indicator of the easy power of today's commodity environments rather than a sign of Silicon Valley's lack of innovation. The numbers are so huge and the valuations are so high we naturally want some sort of fundamental technological revolution to underlie their growth. The revolution is more subtle. It really is just that easy to attain such growth these days, if you can execute on the right idea. Get used to it. This is the new normal.
Here's what Pinterest looks like today:
- 80 million objects stored in S3 with 410 terabytes of user data, 10x what they had in August. EC2 instances have grown by 3x. Around $39K fo S3 and $30K for EC2.
- 12 employees as of last December. Using the cloud a site can grow dramatically while maintaining a very small team. Looks like 31 employees as of now.
- Pay for what you use saves money. Most traffic happens in the afternoons and evenings, so they reduce the number of instances at night by 40%. At peak traffic $52 an hour is spent on EC2 and at night, during off peak, the spend is as little as $15 an hour.
- 150 EC2 instances in the web tier
- 90 instances for in-memory caching, which removes database load
- 35 instances used for internal purposes
- 70 master databases with a parallel set of backup databases in different regions around the world for redundancy
- Written in Python and Django
- Sharding is used, a database is split when it reaches 50% of capacity, allows easy growth and gives sufficient IO capacity
- ELB is used to load balance across instances. The ELB API makes it easy to move instances in and out of production.
- One of the fastest growing sites in history. Cites AWS for making it possible to handle 18 million visitors in March, a 50% increase from the previous month, with very little IT infrastructure.
- The cloud supports easy and low cost experimenation. New services can be tested without buying new servers, no big up front costs.
- Hadoop-based Elastic Map Reduce is used for data analysis and costs only a few hundred dollars a month.
Related Articles
- On Hacker News
- On GigaOM
- Startups Are Creating A New System Of The World For IT
- What is Amazon’s Secret for Success and Why is EC2 a Runaway Train?
Reader Comments (15)
What is the database used for sharding? 70 Masters seems to be higher. Are they in VM? What is the cost involved in keeping the 400+tb data?
"Sites AWS for making it possible"
*Cites
I assume you meant *cites instead of "Sites AWS for making" on the 3rd to last bullet.
Always interesting to see this stuff, 410TB is a huge amount of S3 data!
So, $31K+ / month for storage (and that's reduced redundancy), ~$30K / month for EC2 computation?
So their Amazon bill is roughly $60K/month for EC2 instances and 410TB of storage (not even trying to guess bandwidth) and they have at least another $100K in payroll. They have as far as I can see $0 in revenue. What exactly is the model here?
What is the latest thought on EC2 having pretty bad I/O performance? Does adding more instances make up for this cost-effectively? Thanks.
To improve I/O performance, create multiple volumes and stripe them with software RAID.
Is the ~$69K for storage, per month or per year?
@Joe: Pinterest has experimented with affiliate links in the past, so they've taken in some negligible revenue, but they seem to still be actively figuring it out.
> Here's what Pinterst looks like today:
** Pinterest
Our private cloud team took a look at the above configuration and prices. Based on our experience, private cloud infrastructure would provide a ROI in approximately 20 months. When moving from Amazon to private cloud, we also typically see a 3 to 1 consolidation of instances after migration due to performance increases in I/O and compute power.
So you split up the db it hits 50% of the current server. What do you mean by that? Memory or?
90% of that 410TB likely infringes copyrights. They have absolutely no right to copy fullsize photos to their servers.
Are 18M unique visitors?
They have 152 employes now, What a growth!