Hey, it's HighScalability time: 

 

View of the total solar eclipse from a hill top near Madras Oregon, August 21, 2017. As totality approaches, dragons gorge on sun flesh; darkness cleaves the day; a chill chases away the heat; all becomes still. Contact made! Diamonds glitter; beads sparkle; shadow band snakes slither across pale dust; moon shadow races across the valley, devouring all in wonder. Inside a circle of standing stones, obsidian knives slash and stab. Sacrifices offered, dragons take flight. In awe we behold the returning of the light.

• ~5: ethereum transactions per second; 29+M: Snapchat news viewers; 100K: largest Mastodon instance; 2x: Alibaba's cloud base growth; 1B: trees planted by a province in Pakistan; 90.07%: automated decoding of honey bee waggle dances; $86.4B: Worldwide Information Security Spending; 1200: db migrations from Mysql to Postgres; $7B: Netflix content spend (most not original); 13%: increased productivity by making vacation mandatory; 75%: US teens use iPhones; 30,000x: energy use for Bitcoin transaction compared to Visa; ~1 trillion: observations processed for Gaia mission; 50%: video North American internet traffic; $300 million: cost of cyberattack on world’s biggest container shipping company; 320 million: Freely Downloadable Pwned Passwords; 1700 B.C: world’s oldest trigonometric table;


• Quotable Quotes:
  • @matthew_d_green: I miss the days when Bitcoin was a cool technical innovation and not a weird religious movement.
  • Ruth Williams: Their new digital-to-biological converter (DBC) can, upon receipt of a DNA sequence, prepare appropriate oligos, carry out DNA synthesis, and then, as required, convert that DNA into a vaccine, or indeed into any RNA molecule or protein.
  • @trashcanlife: Hello, this is container 100406100098090 in Buffalo, United States. I am 38% full.
  • Zhang & Stutsman: Developing new systems and applications on RAMCloud, we have repeatedly run into the need to push computation into storage servers.
  • @kevinmontrose: Tomorrow the Sun will undergo routine maintenance in US region. Will be unavailable for select customers, others will have degraded service.
  • @bryanrbeal: We're officially in an era where every piece of HARDWARE you buy, is actually a service. There is no hardware any more.
  • @jessfraz: Literally throwing away two trash bags of container startups tee-shirts, sorry I just... there's too many
  • @postwait: "Scale" I don't think that word means what you think it means. Hint: it doesn't mean your arbitrary concept of "big."
  • Rod Squad: My friend, SocialBlade founder, Jason Urgo advised my 10-year-old son on how to start programming. Jason told us how he started dabbling with scripts and programs in kindergarten. He told me about the first game he programmed. He also listed some of the first applications he built. And he explained how he taught himself PHP to build the YouTube data compiler.
  • Kim Beaudin: Why do Java developers wear glasses? Because they can’t C#.
  • Christine Hall: Investing in private data centers isn’t as much of a priority for IT organizations as it was just several years back. That’s a takeaway from IT researcher Computer Economics’ annual IT Spending and Staffing Benchmarks report...According to the report, data centers now have the lowest priority for new spending among a list of five categories. Top priority is given to the development of business applications, a category in which 54 percent of respondents plan increased spending. However, only 9 percent have plans to increase data center spending, which the study attributes to increasing reliance on cloud infrastructure, cloud storage, and SaaS
  • morning paper: The core idea of a CGN is to gather all the information needed for a page load in a place that has a short RTT time, and then transfer it to the client in (ideally) one round trip. At a cost of about $1 per user, the authors show that it can reduce the median page load time across 100 popular web sites by up to 53%.
  • Nick Harley: It’s easy to shrug off problems with a ‘move fast and break things’ mentality. But we build software for our users, and sometimes forget they are real people.
  • alexkcd: Proof of work systems are, at the core, a race towards ever greater energy consumption. They're an environmental disaster waiting to happen. Surprised how little attention this gets. I would argue that the benefit of decentralization is not worth the price.
  • @EricNewcomer: Uber generates $1.75 billion in revenue on a $645 million loss
  • Preethi Kasireddy: In order to scale, the blockchain protocol must figure out a mechanism to limit the number of participating nodes needed to validate each transaction, without losing the network’s trust that each transaction is valid. 
  • HowDoIMathThough: A slide I personally find really interesting from anandtech's hot chips coverage - Intel has packaging technology that should allow multiple dies to be combined with extremely fast links extremely cheaply
  • Tim Bray: It may sound hack­neyed in 2017, but: Me, I be­lieve in pro­gress. I be­lieve in build­ing un­der­stand­ing cu­mu­la­tive­ly and striv­ing al­ways for Truth. Un­for­tu­nate­ly, there are places in the world, some quite near­by, where the en­e­mies of progress are strong. As Joel Mokyr teach­es, progress is not pre­des­tined to win; we have to fight for it and nev­er stop, or we can lose it; it’s hap­pened.
  • @rawkode: So fed up with watching micro-service talks where they say "More services == good" and don't even mention operational concerns or intg tests
  • @pcalcado: A little known fact is that approximately 47% of CPU usage across a typical Kubernetes cluster is invested translating between JSON and YAML
  • two2two: I asked my 16 year old nephew 6 months ago how he accesses the news. His answer: Snapchat. I followed that with anywhere else? His response was nope.
  • lima: Red Hat's OpenShift makes it [deploying applications?] a lot easier by providing all of the infrastructure around it (docker registry, docker build from Git, Ansible integration and so on). Best docs of all open source projects I've seen.
  • drdaeman: I'm really wary about using larger black boxes for critical parts. Just Linux kernel and Docker can bring enough headache, and K8s on top of this looks terrifying. Simplicity has value. GitHub can afford to deal with a lot of complexity, but a tiny startup probably can't. Or am I just unnecessarily scaring myself?
  • Stefan Majewsky: Across terminals, median latencies ranged between 5 and 45 milliseconds, with the 99.9th percentile going as high as 110 ms for some terminals. Now I can see that more than 100 milliseconds is going to be noticeable, but I was certainly left wondering: Can I really perceive a difference between 5 ms latency and 45 ms latency? Turns out that I can.
  • michaelt: Our current design [for van routing software] isn't well suited to adaption to a GPU, because it branches a lot and the memory accesses aren't strided evenly. So we couldn't just plug our current code into a java-to-cuda compiler; we'd need to change the design.
  • @GabeAul: It's official! We did the last migration this weekend, so all new Windows development is on Git! Congrats to the team who worked the w/end!
  • Iddo Bentov: What we’re seeing today is just a harbinger of problems to come should decentralized exchanges sweep over the cryptocurrency landscape. But since the problems that we’ve identified are exacerbated when higher value trades take place, we conjecture that such problems will ultimately limit the popularity of decentralized exchanges.
  • Steve Goldfeder: trackers can link real-world identities to Bitcoin addresses. To be clear, all of this leaked data is sitting in the logs of dozens of tracking companies, and the linkages can be done retroactively using past purchase data.
  • @jasongorman: Go read somebody else's code, *then* write more unit tests to catch any bugs you find. Code review doesn't scale.
  • David Rosenthal: Unless decentralized technologies specifically address the issue of how to avoid increasing returns to scale they will not, of themselves, fix this economic problem. Their increasing returns to scale will drive layering centralized businesses on top of decentralized infrastructure, replicating the problem we face now, just on different infrastructure.
  • @kevin2kelly: Bill Joy: I decided to spend my time trying to create the things we need as opposed to preventing what threatens us.
  • Ethan Zuckerman: decentralization is important because it allows a community to run under its own rules.
  • Tim Harford: to take advantage of electricity, factory owners had to think in a very different way. They could, of course, use an electric motor in the same way as they used steam engines. It would slot right into their old systems...you couldn't get these results simply by ripping out the steam engine and replacing it with an electric motor. You needed to change everything: the architecture and the production process. And because workers had more autonomy and flexibility, you even had to change the way they were recruited, trained and paid. Factory owners hesitated, for understandable reasons.
  • Dan Luu: We’ve looked at a variety of classic branch predictors and very briefly discussed a couple of newer predictors. Some of the classic predictors we discussed are still used in CPUs today, and if this were an hour long talk instead of a half-hour long talk, we could have discussed state-of-the-art predictors. I think that a lot of people have an idea that CPUs are mysterious and hard to understand, but I think that CPUs are actually easier to understand than software. I might be biased because I used to work on CPUs, but I think that this is not a result of my bias but something fundamental.
  • creshal: A current-gen 35W laptop CPU will be some 10 times faster[2] as a RasPi, have much faster storage available (SATA3 or NVMe versus… USB2), much faster I/O (GBit LAN and GBit Wifi versus… USB2), and a lot of other benefits. (Like an integrated screen and battery and keyboard and …) It also won't need external hardware to communicate with other cluster members – that 10-port ethernet switch will need power, too. One RasPi is relatively energy efficient; RasPi clusters… not so much.
  • howinator: we moved to k8s because we have quite a few low-usage services. Before k8s, each one of those services was getting its own EC2 instance. After k8s, we just have one set of machines which all the services use. If one service is getting more traffic, the resources for that service scale up, but we maintain a low baseline resource usage. In short, it's resulted in a measurable drop in our EC2 usage.
  • medius: If you are migrating to AWS RDS, I recommend AWS Data Migration service. I migrated my live database (~50GB) from Mysql to Postgres (both RDS) with zero downtime. I used AWS Schema Conversion Tool for initial PG schema. I customized the generated schema for my specific needs.
  • Jon Claerbout: interactive programs are slavery unless they include the ability to arrive in any previous state by means of a script
  • @bascule: ~5 transactions/second  @VitalikButerin: Congrats to ethereum community for 5 days of record-high transaction usage! (410061 ... 443356) https://etherscan.io/chart/tx 
  • Sujith Ravi: Delegating the computation-intensive operations from device to the cloud is not a feasible strategy in many real-world scenarios due to connectivity issues (like when data cannot be sent to the server) or privacy reasons. In scenarios, one solution is to take an existing trained neural network model and then apply compression techniques like quantization to reduce model size. The trainer model can be deployed anywhere a standard neural network is used. The simpler projection network model weights along with transform functions are extracted to create a lightweight model that is pushed to device. This model is used directly on-device at inference time.
  • rothbardrand: BCH [Bitcoin Cash] is a hastily written hack job by a third rate team (I talked to some of them on twitter, they really don't understand a lot of what they are doing)... with a drastic difficulty retargeting algorithm. A bit of a pump combined with hash power manipulations lead to this. This is all show to try and prop up the coin. Both the pump and the "profitability" of mining it. %98 of the blocks of this coin are mined by an unknown entity-- in other words, it's not decentralized. It's trivial for that entity to manipulate the difficulty retargeting mechanism in his favor. Stay away. This is not "bitcoin" in any sense.
  • John Allspaw: It’s only when there isn’t universal agreement about a decision (or even if a decision is necessary) that the how, who, and when a decision gets made becomes important to know. The idea of an architecture review is to expose the problem space and proposed departure ideas to dialogue in a broad enough way that confusion about them can be reduced as much as possible. Less confusion about the topic(s) can help reduce uncertainty and/or anxiety about a solution.
  • HBR: Even though the resilient superhero is usually perceived as better, there is a hidden dark side to it: it comes with the exact same traits that inhibit self-awareness and, in turn, the ability to maintain a realistic self-concept, which is pivotal for developing one’s career potential and leadership talent. 
  • Charles Allen: When a local disk fails, the solution is to kill that instance and let the HA built into your application recover on a new VM. When network disk fails or has a multi-instance brownout, you’re just stuck and have to failover to another failure domain, which is usually in another availability zone or in some cases another region! We know this because this kind of failure has caused production outages for us before in AWS. This trend towards network attached storage is one of the scariest industry trends for big data in the cloud where there will probably be more growing pains before it is resolved.

Don't miss all that the Internet has to say on Scalability, click below and become eventually consistent with all scalability knowledge (which means this post has many more items to read so please keep on reading)...