Paper: The Akamai Network - 61,000 servers, 1,000 networks, 70 countries
Thursday, August 18, 2011 at 9:14AM Update: as of the end of Q2 2011, Akamai had 95,811 servers deployed globally.
Akamai is the CDN to the stars. It claims to deliver between 15 and 30 percent of all Web traffic, with major customers like Facebook, Twitter, Apple, and the US military. Traditionally quite secretive, we get a peek behind the curtain in this paper: The Akamai Network: A Platform for High-Performance Internet Applications by Erik Nygren, Ramesh Sitaraman, and Jennifer Sun.
Comprising more than 61,000 servers located across nearly 1,000 networks in 70 countries worldwide, the Akamai platform delivers hundreds of billions of Internet interactions daily, helping thousands of enterprises boost the performance and reliability of their Internet applications. In this paper, we give an overview of the components and capabilities of this large-scale distributed computing platform, and offer some insight into its architecture, design principles, operation, and management.
Delivering applications over the Internet is a bit like living in the Wild West, there are problems: Peering point congestion, Inefficient communications protocols, Inefficient routing protocols, Unreliable networks, Scalability, Application limitations and a slow rate of change adoption. A CDN is the White Hat trying to remove these obstacles for enterprise customers. They do this by creating a delivery network that is a virtual network over the existing Internet. The paper goes on to explain how they make this happen using edge networks and a sophisticated software infrastructure. With such a powerful underlying platform, Akamai is clearly Google-like in their ability to deliver products few others can hope to match.
Detailed and clearly written, it's well worth a read.
Related Articles
- Akamai Technical Publications
- Scaling a Monitoring Infrastructure for the Akamai Network
- When Akamai goes down, it takes the Internet with it by Ryan Lawler
- Akamai Profile on Technology Review
- Akamai's Algorithms
- For all things CDN checkout StreamingMediaBlog by Dan Rayburn
- Inside Akamai and the scary future of streaming video by Stacey Higginbotham
- Microsoft Research Paper Measures Limelight and Akamai's Network Performance by Dan Rayburn
- Akamai: About 3.8 M Simultaneous Obama Streams, Details Capping Of Customers by Dan Rayburn
Reader Comments (4)
A peak or a peek? ;)
A boo on that Michael :-) Thanks, fixed.
From Dan Sweeney:
Nice paper but.. there is a MAJOR FLAW in most CDN providers setups.
The CDN determines which cache site to direct you to based on DNS request from the consumers DNS server. In South East Asia often ISP's don't do DNS well so a large portion of consumers use either Google or OpenDNS as their DNS server. The end result of consumers using Google or OpenDNS is that the CDN directs them NOT TO THE BEST PERFORMANCE cache server.. but to the one their algorythm "thinks" is the best. With less than desirable results in access times.
I proffer the following example:
Sky Broadband AS23944
DNS server 114.108.192.30
DNS server 111.68.59.70
Results:
Sky Broadband's DNS servers think farmville's IP's are:
203.77.188.253
203.77.188.254
These IP's actually belong to a Content Delivery Network provider named
LimeLight using AS22822. I get farmville via their Hong Kong cache.
traceroute to 203.77.188.253 (203.77.188.253), 64 hops max, 52 byte packets
1 192.168.100.1 (192.168.100.1) 1.974 ms 1.297 ms 1.308 ms
2 230-182.gw.skybroadband.com.ph (182.18.230.1) 10.278 ms 10.871 ms 10.952 ms
3 ge-1-4.mnd1.skybroadband.com.ph (111.68.59.141) 36.232 ms 9.665 ms 11.732 ms
4 10ge0-3-0.sj1.skybroadband.com.ph (114.108.192.137) 32.987 ms 26.559 ms
10ge0-3-3.mnd1.skybroadband.com.ph (111.68.59.137) 74.772 ms
5 202.78.101.1 (202.78.101.1) 37.318 ms 61.957 ms 69.786 ms
6 tengige-1-0-0.gw1.rsv.bayan.net.ph (202.78.96.161) 72.134 ms * 96.092 ms
7 limelight1-10g.hkix.net (202.40.161.92) 101.310 ms 110.001 ms 114.412 ms
8 cdn-203-77-188-253.hkg.llnw.net (203.77.188.253) 100.429 ms 83.039 ms 105.583 ms
Change to OpenDNS Server 208.67.222.222
Now my machine thinks Farmville resolves to:
208.111.148.6
208.111.148.7
This is LimeLight CDN AS22822
traceroute to 208.111.148.6 (208.111.148.6), 64 hops max, 52 byte packets
1 192.168.100.1 (192.168.100.1) 1.556 ms 1.277 ms 1.232 ms
2 230-182.gw.skybroadband.com.ph (182.18.230.1) 12.541 ms 11.120 ms 11.883 ms
3 114.108.192.153 (114.108.192.153) 12.625 ms 11.722 ms 11.863 ms
4 114.108.192.133 (114.108.192.133) 40.514 ms 26.772 ms 22.962 ms
5 byt-0027.asianetcom.net (202.147.25.149) 30.663 ms 50.844 ms 28.684 ms
6 po14-0-2.cr1.nrt1.asianetcom.net (202.147.24.222) 80.310 ms 77.755 ms 76.977 ms
7 * po7-0-0.gw1.sjc1.asianetcom.net (202.147.0.34) 186.193 ms 195.764 ms
8 208.178.245.9 (208.178.245.9) 185.150 ms 182.295 ms 196.486 ms
9 64.214.150.94 (64.214.150.94) 200.577 ms 185.450 ms 196.004 ms
10 cdn-208-111-148-6.sjc.llnw.net (208.111.148.6) 198.069 ms 209.952 ms 280.367 ms
Ergo sum
I response to Dan Sweeney's comment.
Using DNS based routing is very simplistic and not the only way to run a CDN.
My understanding is that Akamai uses BGP (border gateway protocol) routing, which tells the network to direct the request for the same IP address to the closest server. So a single IP can be routed to different locations depending on what your ISP's location is.