How Ably Efficiently Implemented Consistent Hashing

This is a guest post by Srushtika Neelakantam, Developer Advovate for Ably Realtime, a realtime data delivery platform.

Ably’s realtime platform is distributed across more than 14 physical data centres and 100s of nodes. In order for us to ensure both load and data are distributed evenly and consistently across all our nodes, we use consistent hashing algorithms.

In this article, we’ll understand what consistent hashing is all about and why it is an essential tool in scalable distributed system architectures. Further, we’ll look at data structures that can be used to implement this algorithm efficiently at scale. At the end, we’ll also have a look at a working example for the same.

Hashing revisited

Remember the good old naïve Hashing approach that you learnt in college? Using a hash function, we ensured that resources required by computer programs could be stored in memory in an efficient manner, ensuring that in-memory data structures are loaded evenly. We also ensured that this resource storing strategy also made information retrieval more efficient and thus made programs run faster.

The classic hashing approach used a hash function to generate a pseudo-random number, which is then divided by the size of the memory space to transform the random identifier into a position within the available space. Something that looked like the following:

location = hash(key) mod size

So, why can’t we use the same method for handling requests over the network?

Click to read more ...


Stuff The Internet Says On Scalability For June 15th, 2018

Hey, it's HighScalability time:


Scaling fake ratings. A 5 star 10,000 phone Chinese click farm. (English Russia)

Do you like this sort of Stuff? Please lend me your support on Patreon. It would mean a great deal to me. And if you know anyone looking for a simple book that uses lots of pictures and lots of examples to explain the cloud, then please recommend my new book: Explain the Cloud Like I'm 10. They'll love you even more.

  • 1.6x: better deep learning cluster scheduling on k8s; 100,000: Large-scale Diverse Driving Video Database; 3rd: reddit popularity in the US; 50%: increase in Neural Information Processing System papers, AI bubble? 420 tons: leafy greens from robot farms; 75%: average unused storage on EBS volumes; 12TB: RAM on new Azure M-series VM; 10%: premium on Google's single-tenant nodes; $7.5B: Microsoft's cost of courting developers; 100th: flip-flop invention anniversary; 1 million: playlist dataset from Spotify; 38GB torrent: Stackoverflow public database; 85%: teens use YouTube; 20%-25%: costs savings using Aurora; 80%: machine learning Ph.D.s work at Google or Facebook; 18: years of NASA satellite data; >1TB: Ethereum blockchain; 200,000 trillion: IBM's super computer calculations per second; 

  • Quotable Quotes:
    • Michael Pollan: “I have no doubt that all that Hubbard LSD all of us had taken had a big effect on the birth of Silicon Valley.”
    • @hisham_hm: Strongly disagree. Most of use who started coding in the 80s started with BASIC. We turned out just fine. The first thing that your first language should teach you is the _joy of coding_.
    • @bryanmikaelian: OH: GraphQL is SOAP for millennials
    • @JoeEmison: Another thing that serverless architectures change: how do you software development. I find myself using a local dev environment to do infra config mgmt, but then often use the web consoles for writing functions and testing; so much faster that way.
    • Jürgen Schmidhuber: And now we can see that all of civilisation is just a flash in world history. In just a flash, the guy who had the first agriculture was almost the same guy who had the first spacecraft in 1957. And soon we are going to have the first AIs that really deserve the name, the first true AIs.
    • Dave Snowden~ A key principle of complex design is shift a system to an adjacent possible. Once there's enough stability more conventional approaches can be used. Architect for discovery before architecting for delivery. Starting with delivery misses the discovery phases which misses opportunities as well as threats. Fractal engagement is they way to achieve change. People don't make decisions about what other people do. People make decision about what they can do tomorrow within their own sphere of influence. The system as a whole orientates through multiple actions. You scale be decomposition and recombination, not through aggregation, and not by imitation.
    • Alex Lindsay: When possibility is greater than circumstance you get action.
    • Andrew Barron: These are all this family of traits that at one time were considered to be the thing that separated humans from all other animals, and then was slowly recognized to appear in primates and then large-brained mammals. And then suddenly we’re recognizing that something like a honey bee, with less than a million neurons, is able to do all of these things.
    • John Hennessy & David Patterson~ We're entering a new golden age [in processors]. The end of Dennard Scaling and Moore's Law means architecture is where we have to innovate to improve performance, cost, and energy. Rasing the level of abstraction using Domain Specific Languages makes it easier for programmers and architects to innovate. Domain Specific Architectures are getting 20x and 40x improvements, not just 5-10%.
    • Hungry for more? Get a plate, click through, and pile it high.

Don't miss all that the Internet has to say on Scalability, click below and become eventually consistent with all scalability knowledge (which means this post has many more items to read so please keep on reading)...

Click to read more ...


Open Source Database HA Resources from Severalnines


Severalnines has spent the last several years writing blogs and crafting content to help make your open source database solutions highly available. We are fans of and wanted to post some links to our top resources to help readers learn more how to make MySQL, MongoDB, MariaDB, Percona and PostgreSQL databases scalable.

Top HA Resources for MySQL & MariaDB

Click to read more ...


Stuff The Internet Says On Scalability For June 8th, 2018

Hey, it's HighScalability time:


Slovenia. A gorgeous place to break your leg. Highly recommended.

Do you like this sort of Stuff? Please lend me your support on Patreon. It would mean a great deal to me. And if you know anyone looking for a simple book that uses lots of pictures and lots of examples to explain the cloud, then please recommend my new book: Explain the Cloud Like I'm 10. They'll love you even more.

  • 294: slides in Internet trends 2018 deck; 110 TB: Hubble Space Telescope data; $124 million: daily App store purchases; 10 billion: monthly Siri requests; 1000 billion: yearly photos taken on iOS; one exabyte: Backblaze storage by year end; 837 million: spam taken down by Facebook in Q1; 86%: of passwords are terrible; 10 Million: US patents; 72: Transceiver Radar Chip; C: most energy efficient language; $138 Billion: global games market; $50 billion: 2017 Angry Birds revenue; 50 million: cesium atomic clock time source; 1.3 Million: vCPU grid on AWS at $30,000 per hour; $296 million: Fortnite April revenue; 4000: Siri requests per second; 

  • Quotable Quotes:
    • @adriancolyer: Microsoft's ServiceFabric runs across over 160K machines and 2.5M cores, powering core Azure services as well as end-user applications. 
    • @StabbyCutyou: To summarize: Tech is not easy, but people are always harder. Until the robots come to replace us all, you won't be able to avoid dealing with people.
      Focus on the above skills, and "technology harder" as a way to increase the scope of where you can be useful to other engineers.
    • Mani Doraisamy: I would rather optimize my code than fundraise.
    • Backblaze: The failure rates of all of the larger drives (8-, 10- and 12 TB) are very good, 1.2% AFR (Annualized Failure Rate) or less. The overall failure rate of 1.84% is the lowest we have ever achieved, besting the previous low of 2.00% from the end of 2017.
    • @rbranson: Things I once held as sincere beliefs: * Databases are bad queue backends * Static typing is a waste of time * Monoliths are always worse * Strong consistency isn't worth fighting for * Using the "right" tool always trumps using the one you know * The JVM sucks
    • @aleksbu: Since I switched back to SSR for my product, productivity went through the roof. SPAs has been a fun journey for me but all this added complexity makes it super expensive. My product doesn’t need any of the benefits that SPAs bring at the moment so switching to SSR was natural.
    • @ajaynairthinks: Real car convo with my 5YO: "V:Papa What's a Lambda" [after hearing my call] Me: Well its a way to run code without servers V: What's code Me: Its like algorithms [from a game he plays] V: Ohh so Lambda is cool because I know what code is and because I dunno what a server is Me: [Yes!]
    • @OpenAI: AI and Compute: Our analysis showing that the amount of compute used in the largest AI training runs has had a doubling period of 3.5 months since 2012 (net increase of 300,000x)
    • @JeffDean: TPUv3!  So hot it needs help with cooling: first liquid cooled devices in our data centers. A TPUv3 pod is 8X as powerful as the TPUv2 pod announced at IO last year, offering more than 100 petaflops of ML compute, allowing us to tackle bigger problems & build better products.
    • Platformonomics: As I keep repeating, CAPEX is both a prerequisite to play in the big boy cloud and confirmation of customer success. Both IBM and Oracle are tens of billions of dollars in cloud infrastructure CAPEX behind Amazon, Google, and Microsoft.
    • Eugene Wei: I believe the core experience of Twitter has reached most everyone in the world who likes it. Let's examine the core attributes of Twitter the product 
    • Harrison Jacobs: The joke used to be that Chinese people like to live near good public schools, Liyan Chen, the manager of international corporate affairs at Alibaba, told Business Insider. “The joke now in China is that they want to live where the Hemas are, because then they can get everything delivered to them really easily.”...The technological advancements Alibaba has brought to Hema — easy in-app ordering, ultrafast delivery, price matching, facial-recognition payment, tailored stocking based on spending habits, etc. — Amazon could easily bring to Whole Foods. And in my opinion, given Amazon's obsession with efficiency, it's a matter of not if, but when.
    • @lizthegrey: Some uses of ML today in Google production: predicting user clicks on ads, prefetching next memory/file accesses, scheduling jobs and capacity planning, speech recognition, fraud detections, smart responses, and machine vision. #SREcon
    • @lizthegrey: What's ML look like in prod? "Don't worry, it's just another data pipeline" 10% of the effort is offline training -- transforming/training prod data using TPUs, validating, and producing a trained model. We then push the trained model to prod. #SREcon
    • Joab Jackson: [Facebook] developed a tool, called Packager, which uses machine learning to automate the process of deciding which files to bundle into a package for a specific end user. It relies heavily on statistical analysis: Which files will the users need right away? Which will they need eventually? Which files have been updated? Some files get updated constantly; others not so much so.
    • It has been a while so you know there's a lot more...

    Don't miss all that the Internet has to say on Scalability, click below and become eventually consistent with all scalability knowledge (which means this post has many more items to read so please keep on reading)...

    Click to read more ...


Gone Fishin'

Well, not exactly Fishin', but I'll be on a month long vacation starting today. I won't be posting new content, so we'll all have a break. Disappointing, I know. Please use this time for quiet contemplation and other inappropriate activities. Se vidiva kmalu!


If you really need a quick fix there's always all the back catalog of Stuff the Internet Says. Odds are there's a lot you didn't read yet.


Stuff The Internet Says On Scalability For April 27th, 2018

Hey, it's HighScalability time:


Did ancient Egyptians invent Wi-Fi? @sherifhanna

Do you like this sort of Stuff? Please lend me your support on Patreon. It would mean a great deal to me. And if you know anyone looking for a simple book that uses lots of pictures and lots of examples to explain the cloud, then please recommend my new book: Explain the Cloud Like I'm 10. They'll love you even more.

  • $17,500: price to give up Google search; 51.8,31.2,18.79: % using AWS, Azure, Google for IoT; 400: items per second shipped by peak Amazon; 43%: music revenues came from streaming; 800%: boost in downloads from apps featured by the Apple App Store; 45: average age of startup founder; 

  • Quotable Quotes:
    • Broad Band: By the mid-twentieth century, computing was so much considered a woman’s job that when computing machines came along, evolving alongside and largely independently from their human counterparts, mathematicians would guesstimate their horsepower by invoking “girl-years,” and describe units of machine labor as equivalent to one “kilogirl.”
    • thegayngler: Most engineers would not hire themselves. That has been apparent to me for awhile now. I’m not sure why they expect people to be to be better than they were when they were hired. I don’t expect engineers to be better than me. I have but one qualification. Can they do the job? Are they strong enough that I can guide them into the position I need them at if it is required.
    • Where Wizards Stay Up Late: Heart liked working with small, tightly knit groups composed of very bright people. He believed that individual productivity and talent varied not by factors of two or three, but by factors of ten or a hundred. Because Heart had a knack for spotting engineers who could make things happen, the groups he had supervised at Lincoln tended to be unusually productive.
    • @AllenDowney: Can someone explain why, if you write an idea in math notation, that's "theory", which provides deep understanding of the math "behind" it, but if you write the same idea in a programming language, it's just hacking? This bizarre prejudice is the bane of my professional life.
    • @kwchang: 'The internet went from a democratizing free space to having power very centralized; crypto decentralization is a reaction to that' - @starkness #Angels #cryptointro
    • Lynn Langit: To me, containers are the new VMs. All this frenzy about containers, and more specifically container management systems — look, somebody has to manage the things. I want to pay the cloud providers to do it so I don’t have to.
    • @kellabyte: Many-core servers are a huge problem. We have no idea how to write software to actually use the hardware properly. For example, in Go, there’s no IO library that can go faster than 10GbE.
    • Markus Winand: Don’t say relational database when referring to SQL databases. SQL is really more than just relational.
    • Catalin Cimpanu: A loud sound emitted by a fire suppression system has destroyed the hard drives of a Swedish data center, downing Nasdaq operations across Northern Europe.
    • dmoy: My mother in law is already paranoid, warning us not to talk about certain topics with her. Scary stuff. I should point out that my MIL is a completely apolitical person who worked in the Chinese government for her entire career, spending most of that time just helping poor people, without a shred of corruption. She has nothing to worry about, doesn't care about politics, and even she's paranoid about this.
    • vkjv: +1 for Neon! The best part about Rust is that it changes this question to "what language AND Rust?" Node + Rust is a great experience.
    • There are lots more quotes. Click through for complete enlightenment.

Don't miss all that the Internet has to say on Scalability, click below and become eventually consistent with all scalability knowledge (which means this post has many more items to read so please keep on reading)...

Click to read more ...


Update: What will programming look like in the future?


UpdateÓlafur Arnalds built a robotic music system to accompany him on the piano. He calls his system of two semi generative, self playing pianos—STRATUS. You can hear his most recent song re:member. There's more explanation in The Player Pianos pt. II and a short Facebook live session. His software reacts to his playing in real-time. Then he has to react to the robots because he's not sure what they're going to do. He's improvising like a jazz band would do, but it's with robots. It's his own little orchestra. The result is beautiful. The result is also unexpected. Ólafur makes the fascinating point that usually your own improvisation is limited by your own muscle memory. But with the randomness of the robots you are forced to respond in different ways. He says you get a "pure unrestricted creativity." And it's fun he says with a big smile on his face.



Maybe programming will look something like the above video. Humans and AIs working together to produce software better than either can separately.

The computer as a creative agent, working in tandem with a human partner, to produce software, in a beautiful act of co-creation.

The alternative vision—The Coming Software Apocalypse—is a dead end.

Better requirements and better tools have already been tried and found wanting. Requirements are a trap. They don't work. Requirements are no less complex and undiscoverable than code.

Tools are another trap. Tools are just code that encode an inflexible solution to a problem that's already been solved.

Admittedly, I'm cheating. I have no idea how any of this will work, but here are the seeds of how it has already started:

Here's what we do know: neither tools or requirements are a silver bullet, they are a method of incrementally improving software quality. Software production quantity is not increased at all.

What we need is a manufacturing process that puts software production on an exponential curve. The only conceivable tool we have at the moment to put software on an exponential production curve is AI. That's the only way software can truly eat the world.

Right now, limited as we are by human programmers using methods that haven't changed much in 30 years, software is just nibbling at the world. And that won't scale. We need more software. A lot more software. And humans are the bottleneck.

Are humans and AIs working together to co-create software the solution? I don't know, but what else is there?


Related Articles


Google: Addressing Cascading Failures

Like the Spanish Inquisition, nobody expects cascading failures. Here's how Google handles them.

This excerpt is a particularly interesting and comprehensive chapter—Chapter 22 - Addressing Cascading Failures—from Google's awesome book on Site Reliability Engineering. Worth reading if it hasn't been on your radar. And it's free!

If at first you don't succeed, back off exponentially."

Dan Sandler, Google Software Engineer

Why do people always forget that you need to add a little jitter?"

Ade Oshineye, Google Developer Advocate

A cascading failure is a failure that grows over time as a result of positive feedback.107 It can occur when a portion of an overall system fails, increasing the probability that other portions of the system fail. For example, a single replica for a service can fail due to overload, increasing load on remaining replicas and increasing their probability of failing, causing a domino effect that takes down all the replicas for a service.

We’ll use the Shakespeare search service discussed in Shakespeare: A Sample Service as an example throughout this chapter. Its production configuration might look something like Figure 22-1.

Example production configuration for the Shakespeare search service.Figure 22-1. Example production configuration for the Shakespeare search service

Causes of Cascading Failures and Designing to Avoid Them

Click to read more ...


Sponsored Post: Datadog, InMemory.Net, Triplebyte, Etleap, Scalyr, MemSQL

Who's Hiring? 

  • Triplebyte lets exceptional software engineers skip screening steps at hundreds of top tech companies like Apple, Dropbox, Mixpanel, and Instacart. Make your job search O(1), not O(n). Apply here.

  • Need excellent people? Advertise your job here! 

Fun and Informative Events

  • Advertise your event here!

Cool Products and Services

  • Datadog is a cloud-scale monitoring platform that combines infrastructure metrics, distributed traces, and logs all in one place. With out-of-the-box dashboards and seamless integrations with over 200 technologies, Datadog provides end-to-end visibility into the health and performance of modern applications at scale. Build your own rich dashboards, set alerts to identify anomalies, and collaborate with your team to troubleshoot and fix issues fast. Start a free trial and try it yourself.

  • InMemory.Net provides a Dot Net native in memory database for analysing large amounts of data. It runs natively on .Net, and provides a native .Net, COM & ODBC apis for integration. It also has an easy to use language for importing data, and supports standard SQL for querying data. http://InMemory.Net
  • For heads of IT/Engineering responsible for building an analytics infrastructure, Etleap is an ETL solution for creating perfect data pipelines from day one. Unlike older enterprise solutions, Etleap doesn’t require extensive engineering work to set up, maintain, and scale. It automates most ETL setup and maintenance work, and simplifies the rest into 10-minute tasks that analysts can own. Read stories from customers like Okta and PagerDuty, or try Etleap yourself.

  • Scalyr is a lightning-fast log management and operational data platform.  It's a tool (actually, multiple tools) that your entire team will love.  Get visibility into your production issues without juggling multiple tabs and different services -- all of your logs, server metrics and alerts are in your browser and at your fingertips. .  Loved and used by teams at Codecademy, ReturnPath, Grab, and InsideSales. Learn more today or see why Scalyr is a great alternative to Splunk.

  • MemSQL envisions a world of adaptable databases and flexible data workloads - your data anywhere in real time. Today, global enterprises use MemSQL as a real-time data warehouse to cost-effectively ingest data and produce industry-leading time to insight. MemSQL works in any cloud, on-premises, or as a managed service. Start a free 30 day trial here:

  • Advertise your product or service here!

If you are interested in a sponsored post for an event, job, or product, please contact us for more information.

Make Your Job Search O(1) — not O(n)

Triplebyte is unique because they're a team of engineers running their own centralized technical assessment. Companies like Apple, Dropbox, Mixpanel, and Instacart now let Triplebyte-recommended engineers skip their own screening steps.

We found that High Scalability readers are about 80% more likely to be in the top bracket of engineering skill.

Take Triplebyte's multiple-choice quiz (system design and coding questions) to see if they can help you scale your career faster.

The Solution to Your Operational Diagnostics Woes

Scalyr gives you instant visibility of your production systems, helping you turn chaotic logs and system metrics into actionable data at interactive speeds. Don't be limited by the slow and narrow capabilities of traditional log monitoring tools. View and analyze all your logs and system metrics from multiple sources in one place. Get enterprise-grade functionality with sane pricing and insane performance. Learn more today

If you are interested in a sponsored post for an event, job, or product, please contact us for more information.


Strategy: Use TensorFlow.js in the Browser to Reduce Server Costs


One of the strategies Jacob Richter describes (How we built a big data platform on AWS for 100 users for under $2 a month) in his relentless drive to lower AWS costs is moving ML from the server to the client.

Moving work to the client is a time honored way of saving on server side costs. Not long ago it might have seemed like moving ML to the browser would be an insane thing to do. What was crazy yesterday often becomes standard practice today, especially when it works, especially when it saves money:

Our post-processing machine learning models are mostly K-means clustering models to identify outliers. As a further step to reduce our costs, we are now running those models on the client side using TensorFlow.js instead of using an autoscaling EC2 container. While this was only a small change using the same code, it resulted in a proportionally massive cost reduction. 

To learn more Alexis Perrier has a good article on Tensorflow with Javascript Brings Deep Learning to the Browser:

Tensorflow.js has four layers: The WebGL API for GPU-supported numerical operations, the web browser for user interactions, and two APIs: Core and Layers. The low-level Core API corresponds to the former deeplearn.js library. It provides hardware-accelerated linear algebra operations and an eager API for automatic differentiation. The higher-level Layers API is used to build machine-learning models on top of Core. The Layers API is modeled after Keras and implements similar functionality. It also allows to import models previously trained in python with Keras or TensorFlow SavedModels and use it for inference or transfer learning in the browser. 

If you want a well paved path to success this is not it. I couldn’t find a lot of success stories yet of using TensorFlow in the browser.

One clear use case came from kbrose:

There are a lot of services that offer free or very cheap hosting of static websites. If you are able to implement your ML model in JS this allows you to deploy your product/app/whatever easily and with low cost. In comparison, requiring a backend server running your model is harder to setup and maintain, in addition to costing more. 

Related Articles