Stuff The Internet Says On Scalability For January 4, 2013

It's HighScalability time:

• 37 Billion: Reddit Pageviews; $3 Billion: Alibaba Sales in a Day; 100 Million: IronMQ Message a Day


• @tomasdev: "Divide and Conquer - The Scalability Technique" seems like divide and conquer is the solution for everything.


• Innovation is all around you. To see it merely open your eyes as much as you've closed your heart.


• Summary of the December 24, 2012 Amazon ELB Service Event in the US-East Region. A developer accidently caused some ELB data to be deleted. It's interesting to consider this incident in the context of resiliency and DevOps. Developers working on production, considered good, we should be full stack, models should be open, etc. Yet Amazon is responding by closing down production systems, just like the bad old days. Couldn't we have our cake and eat it too if the data was backed up and could have been restored quickly? Also, Amazon's EC2 Outage: A Closer Look


• Netflix takes a closer look at the Grinch who stole Christmas eve video: devices streaming when the outage hit continued to stream; many users think the outage was wider than reported; by isolating at the region level other areas of the world were not impacted; regional resiliency is very complicated, but Netflix will look at how to handle region failures.


• Marking the birth of the modern-day Internet which began with the introduction of the Worse is BetterTCP/IP Protocol.


• How low do you want to go? Hot Chips 24 videos are now available. Some of the topics: Mobile SoC Programming, FPGA Augmented ASICs: The Time Has Come, The Future of Wireless Networking, Data Center Chips, and Big Iron. Now that's low level.


• Gartner: 10 Critical Tech Trends For The Next Five Years. Surprising: By 2014, 30% of organizations using SasS will revert back to on-premise due to poor service levels. Not so surprising: SND, BigData, Hybrid Clouds, Client-Server, IoT, virtual appliances, virtual datacenters,  IT demand greater than ever.


• Scalability, Economics, and the Fermi Paradox. John Sloan ponders If Growth is Over? What's next after three successive ways of industrial revolution 1750-1830: steam, railroads; 1870-1900: electricity, internal combustion, running water, indoor toilets, communications, entertainment, chemical, petroleum; 1960-$now: computers, web, mobile. The the problems of scaling up going forward cause humans to extinct themselves?: This doesn't really need to be an NP class problem. It may just be that if a civilization doesn't make getting off of their planet a global priority, the time comes when they no longer have the resources necessary for an interplanetary diaspora or even for interstellar communication.


• Deep in the jungle Netflix finds Janitor Monkey for Keeping the Cloud Tidy and Clean. This monkey sees the wasteful evil of unused resources and cleans them up. One can imagine a playful monkey causing havoc, cleaning up things best left uncleaned, but that's the way it is with monkeys.


• StorageMojo with 2013 predictions: ReRAM hunting for customers that will pay for greater speed, reliability, and durability; drive vendors counting on hybrids to counter the move to mobile; and CIOs who value their jobs will be looking beyond their usual suppliers to better compete with cloud IaaS vendors.


• Node Worker Farm: Distribute processing tasks to child processes with an über-simple API and baked-in durability & custom concurrency options.


• Latency spikes caused by micro-bursts which cause buffer overrun which cause packet drops, even though you have plenty of bandwidth. Witchcraft? Almost. Really good discussion at: TCP incast: What is it? How can it affect Erlang applications?


• What do we do? The mobile future of the Internet will not be monitized: The key driver is that mobile CPMs are only 15 percent of desktop CPMs. As traffic migrates, seven ads on mobile bring the same revenue as one on the desktop, because the lower CPMs coincide with lower click-through rates.


• Adrian Cockcroft Looking back at 2012, with pointers to 2013. Lots of good stuff, but I especially like his observations on how Solar Powered Electrics Cars Are For Real Now. With quality electric cars becoming available and solar panels becominge affordable, it's time to act like the Internet and bypass the Oil Industrial Complex. So: If you don't already have solar panels on your roof, you should get them. If you don't use enough electricity to justify solar panels, get an electric car as well, and save at the gas pump.


• Fault Tolerance via Idempotence: In this paper, we study the inter-related aspects of process failures, duplicate messages, and idempotence. We first introduce a simple core language (based on lambda calculus) inspired by modern distributed computing platforms.


• Don't chuck those smart phones quite yet. Latency – the sine qua non of AR and VR: If you ever thought that AR/VR was just a simple matter of showing an image on the inside of glasses or goggles, I hope that by this point in the blog it’s become clear just how complex and subtle it is to present convincing virtual images.


• Nobody ever got fired for buying a cluster. Real-world analytic jobs process less than 100GB of input which can be processed quite adequately on a large machine. Hadoop doesn't scale-up well so they discuss several scale-up improvements for Hadoop.


• Interesting idea: TCP fast open - sending data with TCP/IP handshake packets. Sort of like a fat message queue notifications that include messages, not just a signal that data is available.  Google found TCP fast open improved page load times by between 4% and 40%.


• Locks under load, always deadly: I discovered to my horror that HashMap has been modified between JDK1.6 and JDK1.7. It now has a block of code that causes all threads creating HashMaps to synchronize. 


• A startup scales up to no avail. Detective Sean Hull finding a serial performance killer, missing indexes and poor SQL. Bigger servers only give a short reprieve, fate comes to all in time, just moving to the cloud is not enough.


• Rob Diana continually surveys the universe for something good for geeks to read. 

Comments

[Firat Atagun]

Divide and conquer is indeed the solution for most problems. Same applies to parallelism and in fact we were thought this approach in introduction to algorithms :)

Sosyal Medya yazilim