Hey, it's HighScalability time:

If you can't explain it with Legos then you don't really understand it.

• 700 trillion: more pixels in Google's Satellite Map; 9,000km: length of new undersea internet cable from Oregon to Japan; 60 terabits per second: that undersea internet cable again; 12%: global average connection speed increase; 76%: WeChat users who spend more than 100RMB ($15) per month; 5 liters: per day pay in beer for Pyramid workers;  680: number of rubber bands it takes to explode a watermelon; 1,000: new Amazon services this year; $15 billion: amount Uber has raised; 7 million: # of feather on on each bird in Piper; 5.8 million: square-feet in Tesla Gigafactory; 2x: full-duplex chip could double phone-network data capacity; 


• Quotable Quotes:
  • @hyc_symas: A shame everyone is implementing on top of HTTP today. Contemporary "protocol design" is a sick joke.
  • @f3ew: Wehkamp lost dev and accept environments 5 days before launch. Shit happens.  48 hours to recovery. #devopsdays
  • Greg Linden: Ultimately, [serverless computing] this is a good thing, making compute more efficient by allowing more overlapping workloads and making it easier to move compute around. But it does seem like compute on demand could cannibalize renting VMs.
  • @viktorklang: What if we started doing only single-core chips with massive eDRAM on-package and PCI-E peer-writes MPI between? (Micro-blade machines?)
  • Robert Graham: Programmers fetishize editors. Real programmers, those who produce a lot of code, fetishize debuggers
  • @jasonhand: "Systems are becoming more ephemeral and we have to find a way to deal with it" (regarding monitoring) - @adrianco #monitorama
  • @aphyr: Queues *can* improve fault tolerance, but this only happens if they don't lose your messages. The only one I know of that doesn't is Kafka.
  • Tom Simon: There are, accordingly, two ways of reading books; but infinitely many ways to divide up the act of reading into two classes.
  • Puppet: High-performing IT organizations deploy 200 times more frequently than low performers, with 2,555 times faster lead times.
  • @benzobot: “The system scaled with the number of engineers - more engineers, more metrics.” #monitorama
  • fizx: You seem to have installation confused with administration. Off the top of my head you forgot security, monitoring, logging config, backups, handling common production issues such as splitbrains, write multiplication, garbage collection snafus, upgrades between versions with questionably compatible internal apis.
  • @Mark_J_Perry: This has to be one of the most remarkable achievements ever: Global Poverty Fell Below 10% for 1st Time in 2015
  • ewams: If you are a services company, he is right, you should be focusing on outcomes. But, if you can't tell me in 2-3 sentences what problem you are solving and how it benefits the customer you are doing it wrong.
  • @retrohack3r: Dance like nobody is watching. Encrypt like everyone is.
  • @GundersenMarius: ES6 + HTTP/2 + Service Workers + Bloom-filter = efficient module loading without bundlin mariusgundersen.net/module-pusher/ 
  • @timperrett: Distributed systems are about protocols, not implementations. Forget languages, protocols are everything.
  • steveblank: What’s holding large companies back?...companies bought into the false premise that they exist to maximize shareholder value – which said “keep the stock price high.” As a consequence, corporations used metrics like return on net assets (RONA), return on capital deployed, and internal rate of return (IRR) to measure efficiency. These metrics make it difficult for a company that wants to invest in long-term innovation.
  • Greg Linden: Like a lot of things at Amazon, this went through many stages of wrong before we got it right. As I remember it, this went through some unpleasant, heavyweight, and inefficient RPC (esp. CORBA) and pub-sub architectures before an unsanctioned skunkworks project built iquitos for lightweight http-based microservices 
  • @rawrthis:  "We all die." Except my legacy stack. That crap will live forever. #devopsdays
  • Jim Handy: The industry already has more than enough DRAM wafer capacity for the foreseeable future. Why is this happening?  The answer is relatively simple: the gigabytes per wafer on a DRAM wafer are growing faster than the market’s demand for gigabytes.
  • SwellJoe: A container doesn't consume complexity and emit order. The complexity is still in there; you still have to build your containers in a way that is replicable, reliable, and automatable. I'm not necessarily saying configuration management is the only way to address that complexity, but it does have to be addressed in a container-based environment.


• Deep Learning desires data, so if you want to build an AI that learns how to program this is how you would go about it, you would bring all the open source code into your giant, voracious, data crunching maw. Making open source data more available: Today, [GitHub] we're delighted to announce that, in collaboration with Google, we are releasing a collection of additional BigQuery tables to expand on the GitHub Archive...This 3TB+ dataset comprises the largest released source of GitHub activity to date. It contains activity data for more than 2.8 million open source GitHub repositories including more than 145 million unique commits, over 2 billion different file paths.


• In almost all cases, the single-threaded implementation outperforms all the others, sometimes by an order of magnitude, despite the distributed systems using 16-128 cores. Scalability! But at what COST? The paper is, at its heart, a criticism of how the performance of current research systems are evaluated. The authors focus on the field of graph processing, but their arguments extend to most distributed computation research where performance is a key factor. They observe that most systems are currently evaluated in terms of their scalability, how their performance changes as more compute resources are used, but that this metric is often both useless and misleading.


• hinkley: The older I get the more I feel like we're an accelerate version of the fashion industry. At least in fashion you can make an excuse that the design is thirty years old and most people don't remember the last time we did this. With software it's every six or seven. It's hard not to judge my peers for having such short memories. We were in the midst of one of these upheavals when I first started, and so I learned programming in that environment. It also means I have one more cycle than most people near my age. Now it all looks the same to me, and I understand those people who wanted to be more conservative. In fact I probably owe some people an apology.


• If you are at all interested in running microservices on Kubernetes then here's the resource for you: Creating a scalable API with microservices. Obviously Google centric. There's a good writeup. You got your video. You got your source code. You got your motivation, being able to write each microservice in a different language yet still have one ops platform. Also, The Children's Illustrated Guide to Kubernetes.


• Peter Norvig with an amazing deep dive on the The Traveling Salesperson Problem. It doubles as a work of art. 


• A new app opportunity. The wisdom of smaller crowds. About the same size that you'll find drinking at a bar. Don't abandon national referendums, but smaller groups often make wiser choices: when it comes to qualitative decisions such as “which candidate will win the election” or “which diagnosis fits the patient’s symptoms,” moderately-sized "crowds," around five to seven randomly selected members are likely to outperform larger ones. In the real world, these moderately-sized crowds manifest as physician teams making medical diagnoses; top bank officials forecasting unemployment, economic growth, or inflation; and panels of election forecasters predicting political wins.


• Thank you ET. A US satellite started transmitting 46 years after being abandoned in 1967. 


• Follow the money. southpolesteve: Define horrific. The traffic that generates those calls also generates lots of revenue :) It is also extremely spiky. This is a big win for Lambda since we are only pay for what we need and don't have to worry about scaling EC2 up and down to meet demand. I have one hard example I can share. We had a node service that was running on ec2 and cost ~$2500/mo. Moved the code directly over to lambda. Now ~$400/mo.


• Infrastructure Software is Dead. Not really. What is defined as infrastructure has just changed. Infrastructure was not dead when people stopped running power plants in their homes to power their electric light bulbs. 


• Braille was the first binary code. Five-Minute Introduction to Braille: "now mainly used by blind people but the original idea was for soldiers to be able to read at night without putting themselves in danger by using any light...The characters of the standard braille alphabet are called cells and consist of [six] dots placed on a two-column grid with three positions in each column; there are 63 different cells not counting the space." Interestingly, like with Unicode points, the meaning of cells is context dependent. You can embed a code to change how later symbols should be interpreted. We can't help but mix the dynamic with our static.


• Good discussion on Why Google Stores Billions of Lines of Code in a Single Repository. It seems people have a hard time groking that what works great for Linux may not work great for Google. Source code is like a Knight's sword, you want to keep it close, secure, and in good working condition at all times, the realm depends on it.


• Ivan Pepelnjak has found a gem. If you don't think you can do what Google does by building your own hardware, here's an inspiring counter example. Build Your Own Service Provider Gear on Software Gone Wild: Initially I thought they built their own Linux distribution on top of x86 server, but what Giacomo Bernardi described in Episode 59 of Software Gone Wild goes way beyond that: Together with an ODM manufacturer they built their own hardware; hey pushed their hardware through rigorous safety, environmental and emission testing – after all, those boxes sit out there in the wild attached to a radio tower; They build their own Linux distribution and hardened it as much as possible – sending a technician to a remote mountain to fix a failed software upgrade is a non-starter; Using 6Wind library they implemented a highly distributed forwarding architecture using dozens of low-cost CPU cores; Together with university researchers they built a central controller that solves the centralized traffic engineering challenge; On top of everything else they implemented their own CLI and network management system.


• War and the organizations capable of fighting them will change radically. Beyond video games: New artificial intelligence beats tactical experts in combat simulation: "ALPHA is so fast it could consider and coordinate the optimal tactical plan and precise responses within a dynamic setting more than 250 times faster than human adversaries can blink." For another kind of war there's Facebook and Google May Be Fighting Terrorist Videos with Algorithms.


• Turning data into music. Melodious it's not, interesting it is. Music of the (data) spheres: The ATLAS experiment has granted MIT Media Lab researchers unique access to a live feed of particle collisions detected at the Large Hadron Collider (LHC)


• Not exactly objective, but you might find the discussion useful. IaaS vs. PaaS vs. CaaS — Which Cloud Architecture is Right For You? Part 1. 


• Keynotes from Velocity in Santa Clara 2016 are now available. Caitie McCaffrey and Ines Sombra have a good 12 minute blitz through a bunch of distributed systems papers that apply to building applications. So we hear you like papers.


• Epic coverage of Apple's new file system by Adam H. Leventhal. A ZFS developer’s analysis of the good and bad in Apple’s new APFS file system: Based on what Apple has shown I'd surmise that its core design goals were: * satisfying all consumers (laptop, phone, watch, etc.) * encryption as a first-class citizen * snapshots for modernized backup. Also, ZFS: The Last Word in File Systems Part 1.


• 3 days of Monitorama live streams are now available online. 


• So they do work. Ish. Amazon expands Dash Buttons into toys, adds 50 brands, says orders have doubled despite skepticism: Amazon this morning said the frequency of orders made through the Dash buttons has doubled in the last three months, now amounting to more than two orders a minute. That translates into 2,880 orders a day, or more than 1 million orders a year.


• Self-driving cars is cool and all, but this is the sort of little problem solving that indicates the kind of changes Deep Learning will make. Bill Slawski: Google has started using machine learning processes to solve problems like customer support. This approach aims at making it easier for people inside of Google to help solve customer problems by better understanding those problems and organizing information about how to solve them. As an SEO, it had me a little excited to see a section that described how Google may rank solutions to problems. This doesn’t appear to be a replacement for PageRank; at least not quite yet. But the roots of organizing a web full of information may be found by starting with solving smaller tasks. 


• A fascinating look at a point-of-sale botnet that appears to have enslaved dozens of hacked payment terminals inside of CiCi’s locations that are being relieved of customer credit card data in real time. Slicing Into a Point-of-Sale Botnet. That botnet's admin panel is certainly enterprise ready.


• Our sense of self is there in the hardware. NeuroLogic: The Brain's Hidden Rationale Behind Our Irrational Behavior: So how does the corollary discharge explain tickling? When you attempt to tickle yourself, a copy of the intended motor action is sent to the sensory system, and the corollary discharge is generated. If the corollary discharge matches the actual sensory experience—the feeling of fingers moving along your ribs at the time and in the pattern that you intended—the brain detects a match and the corollary discharge will dampen the effect of the tickle. In other words, when your brain knows when and how the tickle monster is coming, it’s ready to defend itself. The sensation cancels out, and you don’t get the feeling of tickliness.


• Interesting explanation of Uber Engineering's Micro Deploy: Deploying Daily with Confidence. While you want your microservices to be independent you want a common deployment platform for them all. Their goals: Consistent builds for different services; Zero downtime for upgrades; Early, automated error detection; Outage prevention; Reliable rollouts; Ease of use; REST API for deeper integration. The architecture has a lot in common with monitoring systems with agents, masters, aggregators,and orchestrators.


• We should all be glad these folks are fighting the good fight. How To Break Open The Web: All of us, technically minded or not, need to understand the tradeoffs we’ve been making. Then we need to make decisions. We can accept choke points and lock-in. Or we can look for ways to reclaim control—declining to rely so much on centralized services, and using encryption and the new decentralized tools, such as the already-working IndieWeb, as they become available.


• Backpressure make all the difference. Understanding Akka Streams, Back Pressure and Asynchronous Architecture: You'll be introduced to a number of real life scenarios where applying back-pressure helps to keep your systems fast and healthy at the same time. While the focus is mainly on the Akka Streams implementation, the general principles apply to any kind of asynchronous, message-driven architectures.


• Separate But Equal III: Again: separation of control and data. The old serial port interface is exposed via USB as well, and in fact has its own uses which can be exploited even while a data call is up over the QMI and RMNET channels. But it was another great example of optimizing a control channel for small packets with low latency, and a data channel for big packets with high bandwidth.


• A wonderful look Inside the tiny RFID chip that runs San Francisco's "Bay to Breakers" race: The chip itself is remarkably tiny, about the size of a grain of salt...I took 34 die photos using my metallurgical microscope and stitched them together to get a hi-res photo...The chip has 928 bits of storage (counting up the memory banks on the datasheet) so I suspect the memory is set up as a 32x29 array.


• Fansmitter: Acoustic Data Exfiltration from (Speakerless) Air-Gapped Computers:  We show that a software can regulate the internal fans' speed in order to control the acoustic waveform emitted from a computer. Binary data can be modulated and transmitted over these audio signals to a remote microphone (e.g., on a nearby mobile phone). We present Fansmitter's design considerations, including acoustic signature analysis, data modulation, and data transmission.


• Nike-Inc/wingtips: a distributed tracing solution for Java based on the Google Dapper paper.