Hey, it's HighScalability time:

 

• 50%: drop in latency and CPU load after adopting PHP7 at Tumblr; 4,425: satellites for Skynet; 13%: brain connectome shared by identical twins; 20: weird & wonderful datasets for machine learning; 200 Gb/sec: InfiniBand data rate; 15 TB: data generated nightly by Large Synoptic Survey Telescope; 17.24%: top comments that were also first comments on reddit; $120 million: estimated cost of developing Kubernetes; 3-4k: proteins involved in the intracellular communication network;


• Quotable Quotes:
  • Westworld: Survival is just another loop.
  • Leo Laporte: All bits should be treated equally. 
  • Paul Horner: Honestly, people are definitely dumber. They just keep passing stuff around. Nobody fact-checks anything anymore
  • @WSJ: "A conscious effort by a nation-state to attempt to achieve a specific effect" NSA chief on WikiLeaks 
  • encoderer: For the saas business I run, Cronitor, aws costs have consistently stayed around 10% total MRR. I think there are a lot of small and medium sized businesses who realize a similar level of economic utility.
  • @joshtpm: 1: Be honest: Facebook and Twitter maxed out election frenzy revenues and cracked down once the cash was harvested. Also once political ...
  • boulos: As a counter argument: very few teams at Google run on dedicated machines. Those that do are enormous, both in the scale of their infrastructure and in their team sizes. I'm not saying always go with a cloud provider, I'm reiterating that you'd better be certain you need to.
  • Renegade Facebook Employees: Sadly, News Feed optimizes for engagement. As we've learned in this election, bullshit is highly engaging. A bias towards truth isn't an impossible goal.
  • Russ White: The bottom line is this—don’t be afraid to use DNS for what it’s designed for in your network...We need to learn to treat DNS like it’s a part of the IP stack, rather than something that “only the server folks care about,” or “a convenience for users we don’t really take seriously for operations.”
  • Wizart_App: It's always about speed – never about beauty.
  • Michael Zeltser: MapReduce is just too low level and too dumb. Mixing complex business logic with MapReduce low level optimization techniques is asking too much. 
  • Michael Zeltser: One thing that always bugged me in MapReduce is its inability to reason about my data as a dataset. Instead you are forced to think in single key-value pair, small chunk, block, split, or file. Coming from SQL, it felt like going backwards 20 years. Spark has solved this perfectly.  
  • Guillaume Sachot: I can confirm that I've seen high availability appliances fail more often than non-clustered ones. And it's not limited to firewalls that crash together due to a bug in session sharing, I have noticed it for almost anything that does HA: DRBD instances, Pacemaker, shared filesystems...
  • Albert-Laszlo Barabasi: The bottom line is: Brother, never give up. When you give up, that’s when your creativity ends
  • SpaceX: According to a transcript received by Space News, he argued that the supercooled liquid oxygen that SpaceX uses as propellant actually became so cold that it turned into a solid. And that’s not supposed to happen.
  • Murat: Safety is a system-level property, unit testing of components is not enough.
  • @alexjc: 1/ As deep learning evolves as a discipline, it's becoming more about architecting highly complex systems that leverage data & optimization.
  • btgeekboy: Indeed. If there's one thing I've learned in >10 years of building large, multi-tenant systems, it's that you need the ability to partition as you grow. Partitioning eases growth, reduces blast radius, and limits complexity.
  • @postwait: Monitoring vendors that say they support histograms and only support percentiles are lying to their customers. Full stop. #NowYouKnow
  • @crucially: Fastly hit 5mm request per seconds tonight with a cache hit ratio of 96% -- proud of the team.
  • Rick Webb: Just because Silicon Valley has desperately wanted to believe for twenty years that communities can self-police does not make it true. 
  • Cybiote: Humans can additionally predict other agents and other things about the world based on intuitive physics. This is why they can get on without the huge array of sensors and cars cannot. Humans make up for the lack of sensors by being able to use the poor quality data more effectively. To put this in perspective, 8.75 megabits / second is estimated to pass through the human retina but only on the order of a 100 bits is estimated to reach conscious attention.
  • David Rand: What I found was consistent with the theory and the initial results: in situations where there're no future consequences, so it's in your clear self-interest to be selfish, intuition leads to more cooperation than deliberation.   
  • @crucially: Fastly hit 5mm request per seconds tonight with a cache hit ratio of 96% -- proud of the team
  • SpaceX: With deployment of the first 800 satellites, SpaceX will be able to provide widespread U.S. and international coverage for broadband services. Once fully optimized through the Final Deployment, the system will be able to provide high bandwidth (up to 1 Gbps per user), low latency broadband services for consumers and businesses in the U.S. and globally.
  • Steve Gibson: Anyone can make a mistake [regarding Pixel ownage], and Google is playing security catch up. But what they CAN and SHOULD be proud of is that they had the newly discovered problem patched within 24 hours!
  • dragonnyxx: Calling a 10,000 line program a "large project" is like calling dating someone for a week a "long-term relationship".
  • Brockman: I have three friends: confusion, contradiction, and awkwardness. That’s how I try to meander through life. Make it strange.
  • Martin Sústrik: In this particular case, almost everybody will agree that adding the abstraction was not worth it. But why? It was a tradeoff between code duplication and increased level of abstraction. But why would one decide that the well known cost of code duplication is lower than somewhat fuzzy "cost of abstraction"?


• Biomedical engineering might be an area a lot of tech people interested in real-time monitoring and control at scale could be of help. Hr2: Wireless Spinal Tech, Climate Policy, Moon Impact. Researchers want to use wireless technology to record 100k+ neurons simultaneously, 24x7, for long periods of time. The goal is to use this data to control high dimensional systems, like when when reaching and grasping the shoulder, elbow, hand, wrist, and fingers must all work together in real-time. Sound familiar?


• Making the Switch from Node.js to Golang. Digg switched a S3 heavy service from Node to Go and: Our average response time from the service was almost cut in half, our timeouts (in the scenario that S3 was slow to respond) were happening on time, and our traffic spikes had minimal effects on the service...With our Golang upgrade, we are easily able to handle 200 requests per minute and 1.5 million S3 item fetches per day. And those 4 load-balanced instances we were running Octo on initially? We’re now doing it with 2.


• Not a lie. The best explanation to resilience. Resilience is how you maintain the self-organizing capacity of a system. Great explanation. The way you maintain the resilience of a system is by letting it probe its boundaries. The only way to make forest resilient to fire is to burn it. Efficiency is riding as close as possible to the boundary by using feedback to keep the system self-organizing.


• Facebook does a lot of work making their mobile apps work over poor networks. One change they are making is Client-side ranking to more efficiently show people stories in feed. Previously, all story ranking occurred on the server and entries paged up to the device and displayed in order. The problem with this approach is that an article's rank could change while media is being loaded. Now a pool of stories is kept on the client and as new stories are added they are reranked and shown to users in rank order. This approach adapts well to slow networks because slow-loading content is temporarily down-ranked while it loads.

Don't miss all that the Internet has to say on Scalability, click below and become eventually consistent with all scalability knowledge (which means this post has many more items to read so please keep on reading)...