Hey, it's HighScalability time:

 

• 1+ trillion: messages Twitter handles per day; 695 million: Internet users in China; >350k: Twitter Star Wars bots; $90 million: value of LasVegas.com domain name; 45%: WiFi connection failure rate; 80: threads in Slack Mac OS app; 364: slides in Adrian Cockcroft's microservices deck; 5180%: increases at Etsy in daily visits to pages related to Donald Trump; 465,000: cars sold by Costco last year; 14 Million: one day of DuckDuckGo searches; 58 million: science papers online; ~3x: use of Kubernetes in production settings; 54: r3.2xlarge instances used for Reddit caching; $14 billion: Microsoft’s Azure's annual run rate; 


• Quotable Quotes:
  • Carlo Rovelli: the basic ingredient is down there in the physical world: physical correlation between distinct variables. The physical world is not a set of self-absorbed entities that do their selfish things. It is a tightly knitted net of relative information, where everybody’s state reflects somebody else’s state. 
  • Charles Stross: There’s a saying that goes something like this: “Lieutenants study tactics, colonels study strategy, generals study logistics, and field marshals study economics.” But economists—the smart ones—study education.
  • Kirk Pepperdine: I would suggest that with 200 JVMs running on 80 core you should consider using the serial collector.
  • @alicegoldfuss: Things containers improve: - testing - deploying Things containers shit on: - security - troubleshooting - managing systems resources  Note: this is a long thread of comments, enjoy!
  • @pewinternet: In 2005, just 5% of Americans used at least one social media platform. Today, 69% do. 
  • Manu Saadia: He [Peter Thiel] was a bigger fan of “Star Wars” or “Star Trek,” Thiel replied that, as a capitalist, he preferred the former. “ ‘Star Trek’ is the communist one,” he said. “The whole plot of ‘Star Wars’ starts with Han Solo having this debt that he owes, and so the plot in ‘Star Wars’ is driven by money.
  • @asymco: Google's costs-per-click — essentially its pricing — fell 16% y/y
  • Anna MacLachlan: In order to follow best practices for performance when building PWAs [progressive web app] and otherwise, the Chrome team goes by the Rail performance model: Respond: 100ms / Animate: < 8ms / Idle work in 50ms chunks / Load: 1,000ms to interactive
  • Deepak Singh (AWS): There is a certain scale where specialized hardware and infrastructure make a lot of sense and for those who need special infrastructure, we think FPGAs are one clear way to go
  • @MarcWilczek: Containerization: 19% using it, 15% testing it, 13% considering it; 15% are curious, 38% have no plans or clue. #Cloud #CIO @interop #Docker
  • Clarke Illmatical: The death of net neutrality will severely impact IoT solutions which rely on an open internet concept.
  • @mipearson: OH "I'm the Technical Debt Fairy. If you leave technical debt under your pillowcase at night I hire away your best developers"
  • Reddit:  When you vote, your vote isn’t instantly processed—instead, it’s placed into a queue. Depending on the backlog of the queue, this can mean if you were to vote and quickly refresh the page, your vote may not have been processed yet, and it would appear that your vote had been reverted. 
  • Martin Kleppmann: in a 8,000-node cluster, the chance of permanently losing all three replicas of some piece of data (within the same time period) is about 0.2%. Yes, you read that correctly: the risk of losing all three copies of some data is twice as great as the risk of losing a single node!
  • Tammy Everts: Always remember that if you’re competing online, you’re competing with Amazon.
  • Marco Arment: I'm no spending more on [Apple] search ads than I am servers.
  • dijit: the big issue with databases I've worked with is not how many inserts you do per second, even spinning rust, if properly reasoned can do -serious- inserts per second in append only data structures like myisam, redis even lucene. However the issue comes when you want to read that data or, more horribly, update that data. Updates, by definition are a read and a write to commuted data, this can cause fragmentation and other huge headaches. I'd love to see someone do updates 1,000,000/s
  • @m0biusloop: things kubernetes can't do: ipv6, multiple host networks, prefix based policy, egress policy.
  • Dr Zhou: What is really surprising is our questioning on the whole effort of bot detection in the past years. Suddenly we feel vulnerable and don't know much: how many more are there? What do they want to do?
  • Marianne Bellotti: 15 years ago, everybody was telling us ‘Get off the mainframe, get on AT&T applications, build these thick clients. Mainframes are out.’ And now thick clients are out, and everybody’s moving to APIs and microservices, which basically are very similar to the thin client that a terminal uses to interact with a mainframe.
  • @garybernhardt: Consulting service: you bring your big data problems to me, I say "your data set fits in RAM", you pay me $10,000 for saving you $500,000.
  • @jennschenker: #DLD17: BMW says it will evolve from being a car maker to a mobility services company.
  • Nick Craver (StackOverflow): We try to be boring. Boring is stable ...scalable. The simpler something is, the higher it scales...We are not against anything. We have loyalty to nothing. If there's a better option that comes along, move to it!
  • Romesberg: evolution works by starting with something close, and then changing what it can do in small steps
  • bitwiseand: The CAP theorem states that in the event of a network-partition you have to choose one of C or A. More intuitively, any delay between nodes can be modeled as a temporary network partition and in that event you have but two choices either wait to return the latest data at a peer node (C) or return the last available data at a peer node (A).
  • Gvaireth: We just had a discussion in the team, and we decided, that we need add-one microservice that would get a number and return the number increased by one. A nice separation of concerns in modern distributed web application :)
  • Russ Cox: When I first started thinking about generics for Go in 2008, the main examples to learn from were C#, Java, Haskell, and ML. None of the approaches in those languages seemed like a perfect fit for Go. Today, there are newer attempts to learn from as well, including Dart, Midori, Rust, and Swift.
  • RaptorXP: Do your virtual reality wearables usually connect to deep learning drones on the blockchain?
  • Twitter: Hadoop: We have multiple clusters storing over 500 PB divided in four groups (real time, processing, data warehouse and cold storage). Our biggest cluster is over 10k nodes. We run 150k applications and launch 130M containers per day.
  • arnon: GPUs tend to lend themselves well to analytics, contrary to transactions. Specifically, columnar databases. When the columns are all of the same data type, and the data locality is high, GPUs perform /very/ well.
  • Ed Sim: Despite the amazing productivity gains from open source, AWS, microservices and other new technologies, we have seen the time to launch extending and the cost of getting a minimally viable product (MVP) out the door increasing.
  • Daniel Miessler: It’s [AMP] poisonous to the underlying concept of an open internet. If this were to become widely adopted, you’d search for something, get results, consume the content, and you’d never leave Google.


• Great detailed discussion on all things serverless. AWS Podcast #171: Serverless Special. Serverless is an implementation detail, not an architectural pattern. If you look at serverless as just a way to run existing code then it’s an implementation detail.  If you take it as an opportunity to think about how your application could be structured then it tends more towards the architectural pattern/microservices conversation; Serverless as a concept is a spectrum not binary. Serverless is an important concept but the boundaries are not clear...


• Information wants to be free. Sci-Hub the first pirate website in the world to provide mass and public access to tens of millions of research papers.

Don't miss all that the Internet has to say on Scalability, click below and become eventually consistent with all scalability knowledge (which means this post has many more items to read so please keep on reading)...