Entries by HighScalability Team (1576)

Thursday
Apr092020

Free Google Book: Building Secure and Reliable Systems

 

Google added another book into their excellent SRE series: Building Secure and Reliable Systems. It's free to download, so don't be shy. 

It's not short: 557 pages and 21 chapters! So what's it about? In short it's about "reliability through the lens of security."

In long, Ana Oprea, one of the authors, gave a good overview. anaoprea:

There are multiple questions about what this book is about, who it's for and what might be relevant for me. We recommend going through the Preface to get answers to these questions. Copy/pasting a few paragraphs: "In this book we talk generally about systems, which is a conceptual way of thinking about the groups of components that cooperate to perform some function.

We wanted to write a book that focuses on integrating security and reliability directly into the software and system lifecycle, both to highlight technologies and practices that protect systems and keep them reliable, and to illustrate how those practices interact with each other.

We’d like to explicitly acknowledge that some of the strategies this book recommends require infrastructure support that simply may not exist where you’re currently working.

Because security and reliability are everyone’s responsibility, we’re targeting a broad audience: people who design, implement, and maintain systems. We’re challenging the dividing lines between the traditional professional roles of developers, architects, SREs, systems administrators, and security engineers.

Building and adopting the widespread best practices we recommend in this book requires a culture that is supportive of such change. We feel it is essential that you address the culture of your organization in parallel with the technology choices you make to focus on both security and reliability, so that any adjustments you make are persistent and resilient.

We recommend you start with Chapters 1 and 2, and then read the chapters that most interest you. Most chapters begin with a boxed preface or executive summary that outlines the following:

  • The problem statement
  • When in the software development lifecycle you should apply these principles and practices
  • The intersections of and/or tradeoffs between reliability and security to consider

Within each chapter, topics are generally ordered from the most fundamental to the most sophisticated. We also call out deep dives and specialized subjects with an alligator icon."

I've only had time to browse through the book. One thing I like is they use as examples actual security and design issues they've experienced at Google and then explain how those were solved. Google has problems, just like you.

One thing I always dislike about security books is that empty feeling I get when I go to write that next line of code and I realize I still don't have any idea if that line is secure or not. Even though Chapter 12 is on Writing Code, I'll still get that feeling. Perhaps that's just the nature of the beast. It will take much more practical work before security and reliability can truly become inherent properties of all information systems.

This book is a top to bottom look at a complete organizational response to security and reliability. It's not for the meek:

For those who are passionate about security and reliability, we conclude with the following advice: your ability to work across knowledge domains and embed expertise in the right places is key to your organization’s success. Security and reliability need to be an integrated part of the entire computing environment. All these pieces must work together in harmony to solve problems. No checklist or silver-bullet advice we could give can compensate for your own ability to help your organization flex and grow as the nature of the security and reliability challenges it faces evolves.

Wednesday
Apr082020

One Team at Uber is Moving from Microservices to Macroservices

 

There may be an undiscovered tribe deep in some jungle somewhere that hasn’t made up their mind on microservices, but I doubt it. People love microservices or love to hate microservices. There’s not much in between.

So it means something when even a team at a company like Uber announces a change away from microservices to something else. What? Macroservices. But we’ll get to that. Think what you want about Uber the company, but from a software perspective Uber has been a good citizen.

Gergely Orosz, an Engineering Manager on the Payments Experience Platform at Uber, in a tweet signaled a change in architectural direction:

Click to read more ...

Monday
Mar302020

Post: InterviewCamp.io, Scrapinghub, Fauna, Sisu, Educative, PA File Sight, Etleap, Triplebyte, Stream

Who's Hiring? 

  • InterviewCamp.io has hours of system design content. They also do live system design discussions every week. They break down interview prep into fundamental building blocks. Try out their platform.

  • Scrapinghub is hiring a Senior Software Engineer (Big Data/AI). You will be designing and implementing distributed systems: large-scale web crawling platform, integrating Deep Learning based web data extraction components, working on queue algorithms, large datasets, creating a development platform for other company departments, etc. - this is going to be a challenging journey for any backend engineer! Please apply here

  • Sisu Data is looking for machine learning engineers who are eager to deliver their features end-to-end, from Jupyter notebook to production, and provide actionable insights to businesses based on their first-party, streaming, and structured relational data. Apply here.

  • Triplebyte lets exceptional software engineers skip screening steps at hundreds of top tech companies like Apple, Dropbox, Mixpanel, and Instacart. Make your job search O(1), not O(n). Apply here.

  • Need excellent people? Advertise your job here! 

Cool Products and Services

If you are interested in a sponsored post for an event, job, or product, please contact us for more information.


Essilen Research Free Video Series on Hiring

Modern tech companies understand that hiring is the most important on-going activity they undertake. However, a lot of companies still do their hiring in an ad-hoc way. Thrown-together job specs, random resume screening, unfocused interviews, and a lot more. The result is a process that doesn't get you the best employees you could. World-class tech companies do things very differently. In Essilen Research's video series on tech hiring, you'll learn the secrets that the best companies use to hire the best engineers, developers and designers.

We take you through the hiring process from start to finish. Learn how to write the right job spec. How to screen candidates efficiently, effectively, and without bias. Learn how to train interviewers so everyone's on the same page. No more arguments at the hiring meeting! And there's more: how to offer and close candidates, so they don't slip through your fingers. And of course, how to onboard new-hires the right way so they hit the ground running.

Essilen Research has decades of experience starting, building, running, and scaling great tech organizations. We've conducted thousands of interviews as hiring managers, hired hundreds, and trained dozens of interviewers ourselves. There really IS a better way to do tech hiring and this video series will start you on the path. Level up your tech hiring!


PA File Sight monitors file access on a server in real-time.

It can track who is accessing what, and with that information can help detect file copying, detect (and stop) ransomware attacks in real-time, and record the file activity for auditing purposes. The collected audit records include user account, target file, the user’s IP address and more. This solution does NOT require Windows Native Auditing, which means there is no performance impact on the server. Join thousands of other satisfied customers by trying PA File Sight for yourself. No sign up is needed for the 30-day fully functional trial.


Make Your Job Search O(1) — not O(n)

Triplebyte is unique because they're a team of engineers running their own centralized technical assessment. Companies like Apple, Dropbox, Mixpanel, and Instacart now let Triplebyte-recommended engineers skip their own screening steps.

We found that High Scalability readers are about 80% more likely to be in the top bracket of engineering skill.

Take Triplebyte's multiple-choice quiz (system design and coding questions) to see if they can help you scale your career faster.


If you are interested in a sponsored post for an event, job, or product, please contact us for more information.

Friday
Mar272020

Stuff The Internet Says On Scalability For March 27th, 2020

Hey, it's HighScalability time!

 

Awesome explanation of how to build a PID controller to fly a rocket! (BPS.space via Orbital Index)

 

Do you like this sort of Stuff? Without your support on Patreon this kind of Stuff can't happen. You are that important to the fate of the intelligent world.

Know someone who wants to understand the cloud? I wrote Explain the Cloud Like I'm 10 just for them. On Amazon it has 103 mostly 5 star reviews. Here's a recent authentic unfaked review:

Number Stuff:

Don't miss all that the Internet has to say on Scalability, click below and become eventually consistent with all scalability knowledge (which means this post has many more items to read so please keep on reading)...

Click to read more ...

Tuesday
Mar242020

Post: Scrapinghub, Fauna, Sisu, Educative, PA File Sight, Etleap, Triplebyte, Stream

Who's Hiring? 

  • Scrapinghub is hiring a Senior Software Engineer (Big Data/AI). You will be designing and implementing distributed systems: large-scale web crawling platform, integrating Deep Learning based web data extraction components, working on queue algorithms, large datasets, creating a development platform for other company departments, etc. - this is going to be a challenging journey for any backend engineer! Please apply here

  • Sisu Data is looking for machine learning engineers who are eager to deliver their features end-to-end, from Jupyter notebook to production, and provide actionable insights to businesses based on their first-party, streaming, and structured relational data. Apply here.

  • Triplebyte lets exceptional software engineers skip screening steps at hundreds of top tech companies like Apple, Dropbox, Mixpanel, and Instacart. Make your job search O(1), not O(n). Apply here.

  • Need excellent people? Advertise your job here! 

Cool Products and Services

  • Level up on in-demand technologies and prep for your interviews on Educative.io, featuring popular courses like the bestselling Grokking the System Design Interview. For the first time ever, you can now sign up for a subscription to get unlimited access to every course on the platform at a discounted price through the holiday period only. You'll also get to lock in this price as long as you're a subsciber. 

  • Stateful JavaScript Apps. Effortlessly add state to your Javascript apps with FaunaDB. Generous free tier. Try now!

  • PA File Sight - Actively protect servers from ransomware, audit file access to see who is deleting files, reading files or moving files, and detect file copy activity from the server. Historical audit reports and real-time alerts are built-in. Try the 30-day free trial!

  • Etleap is analyst-friendly, enterprise-grade ETL-as-a-service, built for Redshift and Snowflake data warehouses and S3/Glue data lakes. Our intuitive software allows data engineers to maintain pipelines without writing code, and lets analysts gain access to data in minutes instead of months. Read stories from customers like Okta and PagerDuty, or try Etleap yourself.

  • Build, scale and personalize your news feeds and activity streams with getstream.io. Try the API now in this 5 minute interactive tutorialStream is free up to 3 million feed updates so it's easy to get started. Client libraries are available for Node, Ruby, Python, PHP, Go, Java and .NET. Stream is currently also hiring Devops and Python/Go developers in Amsterdam. More than 400 companies rely on Stream for their production feed infrastructure, this includes apps with 30 million users. With your help we'd like to ad a few zeros to that number. Check out the job opening on AngelList.

  • Advertise your product or service here! 

If you are interested in a sponsored post for an event, job, or product, please contact us for more information.


Essilen Research Free Video Series on Hiring

Modern tech companies understand that hiring is the most important on-going activity they undertake. However, a lot of companies still do their hiring in an ad-hoc way. Thrown-together job specs, random resume screening, unfocused interviews, and a lot more. The result is a process that doesn't get you the best employees you could. World-class tech companies do things very differently. In Essilen Research's video series on tech hiring, you'll learn the secrets that the best companies use to hire the best engineers, developers and designers.

We take you through the hiring process from start to finish. Learn how to write the right job spec. How to screen candidates efficiently, effectively, and without bias. Learn how to train interviewers so everyone's on the same page. No more arguments at the hiring meeting! And there's more: how to offer and close candidates, so they don't slip through your fingers. And of course, how to onboard new-hires the right way so they hit the ground running.

Essilen Research has decades of experience starting, building, running, and scaling great tech organizations. We've conducted thousands of interviews as hiring managers, hired hundreds, and trained dozens of interviewers ourselves. There really IS a better way to do tech hiring and this video series will start you on the path. Level up your tech hiring!


PA File Sight monitors file access on a server in real-time.

It can track who is accessing what, and with that information can help detect file copying, detect (and stop) ransomware attacks in real-time, and record the file activity for auditing purposes. The collected audit records include user account, target file, the user’s IP address and more. This solution does NOT require Windows Native Auditing, which means there is no performance impact on the server. Join thousands of other satisfied customers by trying PA File Sight for yourself. No sign up is needed for the 30-day fully functional trial.


Make Your Job Search O(1) — not O(n)

Triplebyte is unique because they're a team of engineers running their own centralized technical assessment. Companies like Apple, Dropbox, Mixpanel, and Instacart now let Triplebyte-recommended engineers skip their own screening steps.

We found that High Scalability readers are about 80% more likely to be in the top bracket of engineering skill.

Take Triplebyte's multiple-choice quiz (system design and coding questions) to see if they can help you scale your career faster.


If you are interested in a sponsored post for an event, job, or product, please contact us for more information.

Tuesday
Mar172020

Post: Fauna, Sisu, Educative, PA File Sight, Etleap, Triplebyte, Stream

Who's Hiring? 

  • Sisu Data is looking for machine learning engineers who are eager to deliver their features end-to-end, from Jupyter notebook to production, and provide actionable insights to businesses based on their first-party, streaming, and structured relational data. Apply here.

  • Triplebyte lets exceptional software engineers skip screening steps at hundreds of top tech companies like Apple, Dropbox, Mixpanel, and Instacart. Make your job search O(1), not O(n). Apply here.

  • Need excellent people? Advertise your job here! 

Cool Products and Services

  • Level up on in-demand technologies and prep for your interviews on Educative.io, featuring popular courses like the bestselling Grokking the System Design Interview. For the first time ever, you can now sign up for a subscription to get unlimited access to every course on the platform at a discounted price through the holiday period only. You'll also get to lock in this price as long as you're a subsciber. 

  • Stateful JavaScript Apps. Effortlessly add state to your Javascript apps with FaunaDB. Generous free tier. Try now!

  • PA File Sight - Actively protect servers from ransomware, audit file access to see who is deleting files, reading files or moving files, and detect file copy activity from the server. Historical audit reports and real-time alerts are built-in. Try the 30-day free trial!

  • Etleap is analyst-friendly, enterprise-grade ETL-as-a-service, built for Redshift and Snowflake data warehouses and S3/Glue data lakes. Our intuitive software allows data engineers to maintain pipelines without writing code, and lets analysts gain access to data in minutes instead of months. Read stories from customers like Okta and PagerDuty, or try Etleap yourself.

  • Build, scale and personalize your news feeds and activity streams with getstream.io. Try the API now in this 5 minute interactive tutorialStream is free up to 3 million feed updates so it's easy to get started. Client libraries are available for Node, Ruby, Python, PHP, Go, Java and .NET. Stream is currently also hiring Devops and Python/Go developers in Amsterdam. More than 400 companies rely on Stream for their production feed infrastructure, this includes apps with 30 million users. With your help we'd like to ad a few zeros to that number. Check out the job opening on AngelList.

  • Advertise your product or service here! 

If you are interested in a sponsored post for an event, job, or product, please contact us for more information.


Essilen Research Free Video Series on Hiring

Modern tech companies understand that hiring is the most important on-going activity they undertake. However, a lot of companies still do their hiring in an ad-hoc way. Thrown-together job specs, random resume screening, unfocused interviews, and a lot more. The result is a process that doesn't get you the best employees you could. World-class tech companies do things very differently. In Essilen Research's video series on tech hiring, you'll learn the secrets that the best companies use to hire the best engineers, developers and designers.

We take you through the hiring process from start to finish. Learn how to write the right job spec. How to screen candidates efficiently, effectively, and without bias. Learn how to train interviewers so everyone's on the same page. No more arguments at the hiring meeting! And there's more: how to offer and close candidates, so they don't slip through your fingers. And of course, how to onboard new-hires the right way so they hit the ground running.

Essilen Research has decades of experience starting, building, running, and scaling great tech organizations. We've conducted thousands of interviews as hiring managers, hired hundreds, and trained dozens of interviewers ourselves. There really IS a better way to do tech hiring and this video series will start you on the path. Level up your tech hiring!


PA File Sight monitors file access on a server in real-time.

It can track who is accessing what, and with that information can help detect file copying, detect (and stop) ransomware attacks in real-time, and record the file activity for auditing purposes. The collected audit records include user account, target file, the user’s IP address and more. This solution does NOT require Windows Native Auditing, which means there is no performance impact on the server. Join thousands of other satisfied customers by trying PA File Sight for yourself. No sign up is needed for the 30-day fully functional trial.


Make Your Job Search O(1) — not O(n)

Triplebyte is unique because they're a team of engineers running their own centralized technical assessment. Companies like Apple, Dropbox, Mixpanel, and Instacart now let Triplebyte-recommended engineers skip their own screening steps.

We found that High Scalability readers are about 80% more likely to be in the top bracket of engineering skill.

Take Triplebyte's multiple-choice quiz (system design and coding questions) to see if they can help you scale your career faster.


If you are interested in a sponsored post for an event, job, or product, please contact us for more information.

Friday
Mar132020

Stuff The Internet Says On Scalability For March 13th, 2020

p>Hey, it's HighScalability time!

 

The top 10,000 most spoken words in English represented by a point in hundreds of dimensions where the distance and direction between points encodes the relationship between words. (roadmaps)

 

Do you like this sort of Stuff? Without your support on Patreon this kind of Stuff can't happen. You are that important to the fate of the intelligent world.

Know someone who wants to understand the cloud? I wrote Explain the Cloud Like I'm 10 just for them. On Amazon it has 100 (!!!) mostly 5 star reviews. Here's a recent authentic unfaked review:

 

Don't miss all that the Internet has to say on Scalability, click below and become eventually consistent with all scalability knowledge (which means this post has many more items to read so please keep on reading)...

Click to read more ...

Tuesday
Mar032020

Post: Essilen Research, Fauna, Sisu, Educative, PA File Sight, Etleap, Triplebyte, Stream

Who's Hiring? 

  • Are you looking to improve how you hire technical talent? Essilen Research's free video series can help. Fix common issues in screening, interviewing, closing and on-boarding. Learn how world-class tech companies crush the hiring game!

  • Sisu Data is looking for machine learning engineers who are eager to deliver their features end-to-end, from Jupyter notebook to production, and provide actionable insights to businesses based on their first-party, streaming, and structured relational data. Apply here.

  • Triplebyte lets exceptional software engineers skip screening steps at hundreds of top tech companies like Apple, Dropbox, Mixpanel, and Instacart. Make your job search O(1), not O(n). Apply here.

  • Need excellent people? Advertise your job here! 

Cool Products and Services

  • Level up on in-demand technologies and prep for your interviews on Educative.io, featuring popular courses like the bestselling Grokking the System Design Interview. For the first time ever, you can now sign up for a subscription to get unlimited access to every course on the platform at a discounted price through the holiday period only. You'll also get to lock in this price as long as you're a subsciber. 

  • Stateful JavaScript Apps. Effortlessly add state to your Javascript apps with FaunaDB. Generous free tier. Try now!

  • PA File Sight - Actively protect servers from ransomware, audit file access to see who is deleting files, reading files or moving files, and detect file copy activity from the server. Historical audit reports and real-time alerts are built-in. Try the 30-day free trial!

  • Etleap is analyst-friendly, enterprise-grade ETL-as-a-service, built for Redshift and Snowflake data warehouses and S3/Glue data lakes. Our intuitive software allows data engineers to maintain pipelines without writing code, and lets analysts gain access to data in minutes instead of months. Read stories from customers like Okta and PagerDuty, or try Etleap yourself.

  • Build, scale and personalize your news feeds and activity streams with getstream.io. Try the API now in this 5 minute interactive tutorialStream is free up to 3 million feed updates so it's easy to get started. Client libraries are available for Node, Ruby, Python, PHP, Go, Java and .NET. Stream is currently also hiring Devops and Python/Go developers in Amsterdam. More than 400 companies rely on Stream for their production feed infrastructure, this includes apps with 30 million users. With your help we'd like to ad a few zeros to that number. Check out the job opening on AngelList.

  • Advertise your product or service here! 

If you are interested in a sponsored post for an event, job, or product, please contact us for more information.


Essilen Research Free Video Series on Hiring

Modern tech companies understand that hiring is the most important on-going activity they undertake. However, a lot of companies still do their hiring in an ad-hoc way. Thrown-together job specs, random resume screening, unfocused interviews, and a lot more. The result is a process that doesn't get you the best employees you could. World-class tech companies do things very differently. In Essilen Research's video series on tech hiring, you'll learn the secrets that the best companies use to hire the best engineers, developers and designers.

We take you through the hiring process from start to finish. Learn how to write the right job spec. How to screen candidates efficiently, effectively, and without bias. Learn how to train interviewers so everyone's on the same page. No more arguments at the hiring meeting! And there's more: how to offer and close candidates, so they don't slip through your fingers. And of course, how to onboard new-hires the right way so they hit the ground running.

Essilen Research has decades of experience starting, building, running, and scaling great tech organizations. We've conducted thousands of interviews as hiring managers, hired hundreds, and trained dozens of interviewers ourselves. There really IS a better way to do tech hiring and this video series will start you on the path. Level up your tech hiring!


PA File Sight monitors file access on a server in real-time.

It can track who is accessing what, and with that information can help detect file copying, detect (and stop) ransomware attacks in real-time, and record the file activity for auditing purposes. The collected audit records include user account, target file, the user’s IP address and more. This solution does NOT require Windows Native Auditing, which means there is no performance impact on the server. Join thousands of other satisfied customers by trying PA File Sight for yourself. No sign up is needed for the 30-day fully functional trial.


Make Your Job Search O(1) — not O(n)

Triplebyte is unique because they're a team of engineers running their own centralized technical assessment. Companies like Apple, Dropbox, Mixpanel, and Instacart now let Triplebyte-recommended engineers skip their own screening steps.

We found that High Scalability readers are about 80% more likely to be in the top bracket of engineering skill.

Take Triplebyte's multiple-choice quiz (system design and coding questions) to see if they can help you scale your career faster.


If you are interested in a sponsored post for an event, job, or product, please contact us for more information.

Friday
Feb282020

Stuff The Internet Says On Scalability For February 28th, 2020

Wake up! It's HighScalability time:

 

The Smithsonian has million of pieces of delicious open access content. I ate up this 3D representation of the 1903 Wright Flyer

 

Do you like this sort of Stuff? Without your support on Patreon this kind of Stuff won't happen. That's how important you are to the fate of the world.

Need to understand the cloud? Know someone who does? I wrote Explain the Cloud Like I'm 10 just for you...and them. On Amazon it has 98 mostly 5 star reviews. Here's a recent authentic unfaked review:

 

Number Stuff: 

Don't miss all that the Internet has to say on Scalability, click below and become eventually consistent with all scalability knowledge (which means this post has many more items to read so please keep on reading)...

Click to read more ...

Monday
Feb242020

Socratic vs. Euclidean Forms of API Documentation

 

I was emailing a service about their documentation and while their doc was good, about one particularly tricky concept they told me that once you use it for a while, that’s when you’ll understand it.

In other words: you’ll only understand it after you understand it.

I didn’t like that response. I want documentation that takes me from an unproductive newbie to a somewhat functioning journeyperson. Not an expert, but I want to get stuff done as soon as possible. And for that you need to understand the mental model behind the API. Otherwise, how do you know how to make anything happen?

I realize it’s hard to make good documentation. I spent a lot of time writing Explain the Cloud Like I'm 10 just to communicate the mental model behind the cloud. It’s not easy.

Then I read that something that showed me there are two different styles of documentation: Euclidean and Socratic:

Euclidean - state your axioms and let users derive the rest. Easiest for the API provider, but hardest on the user. This is the most common form of documentation. You see it all the time. Each entry point in the API is sort of explained, but there’s nothing tying the whole API together. You just pray someone on Stackoverflow already asked the questions you want to ask and someone made the effort to answer—before the question was voted into oblivion.

Socratic - an open-inquiry meant to bring about a deeper understanding in the API user. You get the Euclidean part, but you also get a FAQ, you get recipes for common tasks, you get error conditions and possible responses, and you get working code examples. You get a deep explanation of what the API is trying to accomplish and how you can use it to accomplish your own goals. People tend to think once they've written an example on GitHub that they've fulfilled their Socratic duty. Not so. You need to help people get to the point where they could write the example code. The person who works at the company and wrote the example already knows all that, but it's that knowledge that must be communicated, not the end product.

API doc tends to be Euclidean, but at its best documentation is Socratic. That’s when you can really be productive—fast.

If you can't explain something well, it's likely you don't understand it either. And if you don't understand it as an API provider, how is anyone else going to understand it? Please make that extra effort. 

What did I read that explained the idea of Euclidean vs Socratic approaches to a topic? Take a look at this interview with David B. Kinney on the Philosophy of Science:

Click to read more ...