Entries by HighScalability Team (1576)

Tuesday
Jul122011

Google+ is Built Using Tools You Can Use Too: Closure, Java Servlets, JavaScript, BigTable, Colossus, Quick Turnaround

Joseph Smarr, former CTO of Plaxo (which explains why I recognized his picture), in I'm a technical lead on the Google+ team. Ask me anything, reveals the stack used for building Google+:

Click to read more ...

Tuesday
Jul122011

Sponsored Post: New Relic, eHarmony, TripAdvisor, NoSQL Now!, Surge, BioWare, Tungsten, deviantART, Aconex, Hadapt, Mathworks, AppDynamics, ScaleOut, Membase, CloudSigma, ManageEngine, Site24x7

Who's Hiring?

  • TripAdvisor is Hiring Engineers at all Levels: Scalable Web Engineering Program. To apply for our Scalable Web Engineering Program, visit http://www.tripadvisor.com/careers/webprogram
  • Are you a scalability expert? eHarmony is looking for Senior Java Engineers to help implement and scale our Matching compatibility systems. Please visit: http://tinyurl.com/3g8mxks.
  • BioWare Austin is looking for a Performance Test Engineer for our Austin team. To apply, please visit http://www.bioware.com/careers/austin.
  • BioWare Austin is looking for a Contract Build Engineer for our Austin team. To apply, please visit http://www.bioware.com/careers/austin.
  • deviantART is looking for Network and Systems Operations Engineer. Please apply here.
  • Aconex is looking for a Systems Engineer in San Bruno. Please apply here.
  • Hadapt brings high-performance SQL to Hadoop, and is looking for a systems engineer to join this fast-growing company. Please apply at http://www.hadapt.com/jobs.
  • MathWorks Looking for Multiple, Full-time Scaling Experts. Apply now: http://matlab.my/lVmunb 

Fun and Informative Events

  • NoSQL Now! is a new conference covering the dynamic field of NoSQL technologies. August 23-25 in San Jose. For more information please visit: http://www.NoSQLNow.com
  • Surge 2011: The Scalability and Performance Conference. Surge is a chance to identify emerging trends and meet the architects behind established technologies. Early Bird Registration.
  • Join our webinar as we introduce Tungsten Enterprise Summer '11 Edition with improved usability, performance and ease of management for MySQL and PostgreSQL clusters.
  • Couchbase is having a Special Offer for Apache CouchDB Developer Training! http://www.couchbase.com/couchdb-training/portland-june-2011

Cool Products and Services

  • New Relic - real user monitoring optimize for humans, not bots. Live application stats, SQL/NoSQL performance, web transactions, proactive notifications. Take 2 minutes to sign up for a free trial.
  • AppDynamics is the very first free product designed for troubleshooting Java performance while getting full visibility in production environments. Visit http://www.appdynamics.com/free.
  • ScaleOut StateServer - Scale Out Your Server Farm Applications! 
  • CloudSigma. Instantly scalable European cloud servers.
  • ManageEngine Applications Manager : Monitor physical, virtual and Cloud Applications.
  • www.site24x7.com : Monitor End User Experience from a global monitoring network.

For a longer description of each sponsor, please read more below...

Click to read more ...

Monday
Jul112011

ATMCash Exploits Virtualization for Security - Immutability and Reversion

This is a guest post by Ran Grushkowsky, Head of Technology at ATMCash.

Virtualization and cloud-based systems are very hype in the industry; however, most financial companies stray from those solutions. At ATMCash, we’ve approached virtualization not for the usual reason of scalability, but for the usually missed value of security.

In this article, I will introduce the concept of security added value in the utilization of virtualization and why people should consider deploying mini-clouds for those use-cases. 

How do virtual machines help mitigate risk?

I am sure most of you have heard of the recent hacking in the financial sector. Financial companies are under constant hacking attempts and security is of the utmost importance. One of the bigger risks in system deployment is a breach in one of the stack components. Regular system patches and maintenance fix known exploits and issues, however, sometimes it may be too late and the component has already been breached. If the system has already been compromised in a natural environment where patches are applied to existing systems, sometimes the patch may come too late and a Trojan horse or some sort of malicious code has already been injected (as may have been seen in recent cases). Virtual machines provide a great hidden gem: immutability and reverting images

Example of how ATMCash uses those features for security in the stack:

Click to read more ...

Friday
Jul082011

Stuff The Internet Says On Scalability For July 8, 2011

Submitted for your scaling pleasure: 

For even more Stuff the Internet says, please read more below...

Click to read more ...

Thursday
Jul072011

Myth: Google Uses Server Farms So You Should Too - Resurrection of the Big-Ass Machines

For a long epoch there was a strategy of scaling up by making ever bigger super computers. I had the pleasure of programming on a few large massively multi-processor machines from SGI and DEC. Beautiful, highly specialized machines that were very expensive. These met the double-tap extinction event of Moore's law and a Google inspired era of commodity machine based clusters and extreme software parallelism. Has the tide turned? Does it now make more sense to use big machines instead of clusters?

In Big-Ass Servers™ and the myths of clusters in bioinformatics, Jerm makes the case that for bionformatics, it's more cost effective to buy a Big-Ass Server instead of using a cluster of machines and a lot of specialized parallel programming techniques. It's a classic scale-up argument that has been made more attractive by the recent development of relatively inexpensive large machines. SeaMicro has developed a 512 core machine. Dell has a new 96 core server. Supermicro has 48 core machines. These are new options in the scale-up game that have not been available before and could influence your architecture choice.

Jerm's reasoning for preferring big-ass servers is:

Click to read more ...

Tuesday
Jul052011

Sponsored Post: TripAdvisor, eHarmony, NoSQL Now!, Surge, BioWare, Tungsten, deviantART, Aconex, Hadapt, Mathworks, AppDynamics, ScaleOut, Membase, CloudSigma, ManageEngine, Site24x7

Who's Hiring?

  • TripAdvisor is Hiring Engineers at all Levels: Scalable Web Engineering Program. To apply for our Scalable Web Engineering Program, visit http://www.tripadvisor.com/careers/webprogram
  • Are you a scalability expert? eHarmony is looking for Senior Java Engineers to help implement and scale our Matching compatibility systems. Please visit: http://tinyurl.com/3g8mxks.
  • BioWare Austin is looking for a Performance Test Engineer for our Austin team. To apply, please visit http://www.bioware.com/careers/austin.
  • BioWare Austin is looking for a Contract Build Engineer for our Austin team. To apply, please visit http://www.bioware.com/careers/austin.
  • deviantART is looking for Network and Systems Operations Engineer. Please apply here.
  • Aconex is looking for a Systems Engineer in San Bruno. Please apply here.
  • Hadapt brings high-performance SQL to Hadoop, and is looking for a systems engineer to join this fast-growing company. Please apply at http://www.hadapt.com/jobs.
  • MathWorks Looking for Multiple, Full-time Scaling Experts. Apply now: http://matlab.my/lVmunb 

Fun and Informative Events

  • NoSQL Now! is a new conference covering the dynamic field of NoSQL technologies. August 23-25 in San Jose. For more information please visit: http://www.NoSQLNow.com
  • Surge 2011: The Scalability and Performance Conference. Surge is a chance to identify emerging trends and meet the architects behind established technologies. Early Bird Registration.
  • Join our webinar as we introduce Tungsten Enterprise Summer '11 Edition with improved usability, performance and ease of management for MySQL and PostgreSQL clusters.
  • Couchbase is having a Special Offer for Apache CouchDB Developer Training! http://www.couchbase.com/couchdb-training/portland-june-2011

Cool Products and Services

For a longer description of each sponsor, please read more below...

Click to read more ...

Friday
Jul012011

TripAdvisor Strategy: No Architects, Engineers Work Across the Entire Stack

If you are an insect, don't work at TripAdvisor, specialization is out. One of the most commented on strategies from the TripAdvisor architecture article is their rather opinionated take on the role of engineers in the organization.

Typically engineers live in a box. They are specialized, they do database work and not much else, and they just do programming, not much else. TripAdvisor takes the road less traveled:

  • Engineers work across entire stack - HTML, CSS, JS, Java, scripting. If you do not know something, you learn it. The only thing that gets in the way of delivering your project is you, as you are expected to work at all levels - design, code, test, monitoring, CSS, JS, Java, SQL, scripting.
  • We do not have "architects."  At TripAdvisor, if you design something, your code it, and if you code it you test it. Engineers who do not like to go outside their comfort zone, or who feel certain work is "beneath" them will simply get in the way.

A radical take for an established organization. We've seen companies where backend engineers are also responsible for front-line site support, but I don't recall, other than for startups obviously, where engineers have such breadth and depth of responsibility. It's not right or wrong, but something outside-the-box to consider, especially if you are an Agile shop.

Friday
Jul012011

Stuff The Internet Says On Scalability For July 1, 2011

Submitted for your scaling pleasure: 

  • Twitterers tweet 200 million tweets a day. Popular topics are eclectic, ranging from Swine Flu to Rebecca Black. Twitter has a really cool video on the global flow of tweets in the world. Worth watching. It looks like a rainbow arcing across the northern hemisphere.
  • Amazon Cloud Now Stores 339 Billion Objects, more than doubling last years volume. 
  • Quotable quotes for independence Alex:
    • n8foo: My fav part about the new #AWS pricing announcement - 500TB is the level where they say 'contact us'.
    • stevedekorte: Are the folks advocating FP for scalability unaware of the Von Neumann bottleneck?
    • lhazlewood: I don't think I've ever had a Love/Hate relationship like I've had with NoSQL. It's all awesome. And it all sucks.
  • Free is good says Amazon, No Inbound Data Transfer Fees and slightly lower outbound fees. Will Amazon make up for this with voume? Maybe S3 sales (which are not cheaper)? it's true, most sites upload little data compared to their outbound traffic, but for some use cases it's a real win, email, scraper bots, chat, video, and backup.
For lots more Stuff the Internet said, please read below...

Click to read more ...

Wednesday
Jun292011

Second Hand Seizure : A New Cause of Site Death

Like a digital SWAT team that implodes the wrong door on a raid, the FBI seized multiple racks of computers from DigitalOne, these racks host websites from many clients that just happened to be in the same racks as whomever they are investigating. Downed sites include Instapaper, Curbed Network, and Pinboard. With the density of servers these days many 1000s of sites could easily have been effected.

Sites like Pinboard were victims by association, they did not inhale. This is an association sites have no control over. On a shared hosting service, you have no control over your fellow VM mates. In a cloud or a managed service, you have no control over which racks your servers are in. So like second hand smoke, you get the disease by random association. There's something inherently unfair about that.

A comment by illumin8 shows just how Darth insidious this process can be:

Click to read more ...

Tuesday
Jun282011

Sponsored Post: Surge, BioWare, Tungsten, deviantART, Aconex, Hadapt, Mathworks, AppDynamics, ScaleOut, Membase, CloudSigma, ManageEngine, Site24x7

Who's Hiring?

  • BioWare Austin is looking for a Performance Test Engineer for our Austin team. To apply, please visit http://www.bioware.com/careers/austin.
  • BioWare Austin is looking for a Contract Build Engineer for our Austin team. To apply, please visit http://www.bioware.com/careers/austin.
  • deviantART is looking for Network and Systems Operations Engineer. Please apply here.
  • Aconex is looking for a Systems Engineer in San Bruno. Please apply here.
  • Hadapt brings high-performance SQL to Hadoop, and is looking for a systems engineer to join this fast-growing company. Please apply at http://www.hadapt.com/jobs.
  • MathWorks Looking for Multiple, Full-time Scaling Experts. Apply now: http://matlab.my/lVmunb 

Fun and Informative Events

Cool Products and Services

For a longer description of each sponsor, please read more below...

Click to read more ...