Google: Multiplex Multiple Works Loads on Computers to Increase Machine Utilization and Save Money
Wednesday, November 13, 2013 at 8:54AM
Wednesday, November 13, 2013 at 8:54AM
Tuesday, November 12, 2013 at 8:54AM 
If any of these items interest you there's a full description of each sponsor below. Please click to read more...
Monday, November 11, 2013 at 8:54AM 
This question was asked over email and I thought a larger audience might want to take a whack at it.
With a business associate, I am trying to develop a financial software that handles financial reports of listed companies. We managed to create this database with all the data necessary to do financial analysis. My associate is a Business Intelligence specialist so he is keen to use OLAPs databases like Microsoft Analysis Services or Jedox Palo, which enables in-memory calculations and very fast aggregation, slicing and dicing of data or write-backs.
At the same time I did an online course (MOOC) from Stanford CS184 called Startup Engineering which promoted/talked a lot about javascript and especially node.js as the language of the future for servers.
As I am keen to use open-source technologies (would be keen to avoid MS SSAS) for the development of a website to access this financial data , and there are so many choices for databases out there (Postgre, MongoDB, MySQL etc..but don't think they have OLAP features), do you know of resources, blogs, people knowledgeable on the matter, which talk about combination of node.js with OLAP databases? best use of a particular system with node.js?
Thanks for your input.
Friday, November 8, 2013 at 8:54AM Hey, it's HighScalability time:
Don't miss all that the Internet has to say on Scalability, click below and become eventually consistent with all scalability knowledge...
Thursday, November 7, 2013 at 8:55AM 
An interesting and different implementation approach: Tempest: Scalable Time-Critical Web Services Platform:
Tempest is a new framework for developing time-critical web services. Tempest enables developers to build scalable, fault-tolerant services that can then be automatically replicated and deployed across clusters of computing nodes. The platform automatically adapts to load fluctuations, reacts when components fail, and ensures consistency between replicas by repairing when inconsistencies do occur. Tempest relies on a family of epidemic protocols and on Ricochet, a reliable time critical multicast protocol with probabilistic guarantees.
Tempest is built around a novel storage abstraction called the TempestCollection in which application developers store the state of a service. Our platform handles the replication of this state across clones of the service, persistence, and failure handling. To minimize the need for specialized knowledge on the part of the application developer, the TempestCollection employs interfaces almost identical to those used by the Java Collections standard. Elements can be accessed on an individual basis, but it is also possible to access the full set by iterating over it, just as in a standard Collection. The hope is that we can free developers from the complexities of scalability and fault-tolerance, leaving them to focus on application functionality.
Traditionally, services relying on a transactional database backend offer a strong data consistency model in which every read operation returns the result of the latest update that occurred on a data item. With Tempest we take a different approach by relaxing the model such that services offer sequential consistency [10]: Every replica of the service sees the operations on the same data item in the same order, but the order may be different from the order in which the operations were issued. Later, we will see that this is a non-trivial design decision; Tempest services can sometimes return results that would be erroneous were we using a more standard transactional execution model. For applications where these semantics are adequate, sequential consistency buys us scheduling flexibility that enables much better real-time responsiveness.
Tuesday, November 5, 2013 at 9:09AM
Authored by Chris Fregly: Former Netflix Streaming Platform Engineer, AWS Certified Solution Architect and Purveyor of fluxcapacitor.com.
Ahead of the upcoming 2nd annual re:Invent conference, inspired by Simone Brunozzi’s recent presentation at an AWS Meetup in San Francisco, and collected from a few of my recent Fluxcapacitor.com consulting engagements, I’ve compiled a list of 10 useful time and clock-tick saving tips about AWS.
1) Query AWS resource metadata
Can’t remember the EBS-Optimized IO throughput of your c1.xlarge cluster? How about the size limit of an S3 object on a single PUT? awsnow.info is the answer to all of your AWS-resource metadata questions. Interested in integrating awsnow.info with your application? You’re in luck. There’s now a REST API, as well!
Note: These are default soft limits and will vary by account.
2) Tame your S3 buckets
Delete an entire S3 bucket with a single CLI command:
aws s3 rb s3://<bucket-name> --force
Recursively copy a local directory to S3:
aws s3 cp <local-dir-name> s3://<bucket-name> --region <region-name> --recursive
3) Understand AWS cross-region dependencies
Amazon,
Cloud Computing,
Distributed Computing,
Scalability,
amazon ec2,
architecture,
aws,
public cloud,
virtualization in
AWS,
Amazon S3,
EC2,
S3,
ZFS,
amazon,
cloud computing,
distributed systems,
netflix Tweet
Monday, November 4, 2013 at 8:52AM 
ESPN went on air in 1978. In those 30+ years think of the wonders we’ve seen! When I think of ESPN I think of a world wide brand that is the very definition of prime time. And it shows in their stats. ESPN.com peaks at 100,000 requests per second. Their peak event is, not surprisingly, the World Cup. But would you be surprised to learn ESPN is powered by only a few hundred servers and a couple of dozen engineers? I was.
And would you be surprised to learn ESPN is undergoing a fundamental transition from an Enterprise architecture to one capable of handling web scale loads driven by increasing mobile usage, personalization, and a service orientation? Again, thinking ESPN was just about watching sports on TV, I was surprised. ESPN is becoming much more than that. ESPN is becoming a sports platform.
How does ESPN handle all of this complexity, responsibility, change, and load? Unlike most every other profile on HighScalability. The fascinating story of ESPN’s architecture is told by Manny Pelarinos, Senior Director, Engineering at ESPN in the InfoQ presentation Architecture at Scale at ESPN. Information from Max Protect: Scalability and Caching at ESPN.com has also been folded in.
Starting in a pre-personal computer era ESPN developed an innovative cable and satellite TV sports empire. From an initial 30 minute program reviewing the day’s sports, they went on to make deals with the NBA, USFL, NHL, and what would become the big fish of all sports in the US, the National Football League.
Sport by sport deals were made to bring sports data in from all possible sources so ESPN could report scores, play film clips, and generally become one stop shopping for all things sports on TV and later the web.
It’s a complex system to understand. They have a lot going on with Television & Broadcasting, live scoring, editing and publishing, Digital Media, giving sports scores, web and mobile, personalization, fantasy games, and they also want to expand API access to 3rd party developers. Unlike most every profile on HighScalability ESPN has an enterprise heritage. It’s a Java Enterprise stack, so you’ll see Oracle databases, JMS brokers, Java Beans, and Hibernate.
Some of the most important lessons we’ll learn about:
Platform changes everything. ESPN sees themselves as a content provider. These days content is accessed through multiple paths. It can be on TV, or on ESPN.com, or on mobile, but content is also being consumed by more and more internal applications, like Fantasy Games. And they also want to provide an external API so developers can build on ESPN resources. ESPN wants to become a walled garden built on a sports content platform that centralizes access to their prime advantage over everyone else, which is unprecedented access to sports related content and data. The walled garden approach that Facebook has made work for social, Apple has made work for apps, and Google has made work for AI, is what ESPN wants to do for sports. The problem is transitioning from an enterprise architecture to a platform based on APIs and services is a tough change to make. They can do it. They are doing it. But it will be hard.
Web scale changes everything. Many web properties today use Java as their standard backend development environment, but ESPN.com, which grew up in the Java Enterprise era, went all in for the canonical Enterprise architecture. And it has worked quite well. Until there was a sort of phase transition from enterprise class loads experienced by a relatively predictable ESPN.com to a world dominated by high mobile traffic, mass customization, and platform concerns. Many of the architecture choices we see in native web properties must now be used by ESPN.com.
Personalization changes everything. The cache that once saved your database is now much less useful when all content becomes dynamically constructed for each user and must follow you on every mode of access (.com, mobile, TV).
Mobile changes everything. It puts pressure everywhere on your architecture. When there was just the web architecture didn’t matter as much because there were fewer users and fewer servers. In the mobile age with so many more users and servers these kind of architecture decisions make a huge difference.
Partnerships are power. ESPN can create a walled garden because over the years they have developed partnerships that gives them special access to data that nobody else has. It’s good to be firstest with the mostest. That individual sports like the NFL and MLB seeking to capture this value with their own network lessens this advantage somewhat, but the forces are such that everyone needs to get along, which puts ESPN in the middle of a powerful platform play, if they can execute.
Lights. Camera. Action. Let’s learn how ESPN scales...
Thursday, October 31, 2013 at 8:55AM 
Awesome paper on how particular synchronization mechanisms scale on multi-core architectures: Everything You Always Wanted to Know About Synchronization but Were Afraid to Ask.
The goal is to pick a locking approach that doesn't degrade as the number of cores increase. Like everything else in life, that doesn't appear to be generically possible:
None of the nine locking schemes we consider consistently outperforms any other one, on all target architectures or workloads. Strictly speaking, to seek optimality, a lock algorithm should thus be selected based on the hardware platform and the expected workload.
Abstract:
Wednesday, October 30, 2013 at 8:55AM Oh, you don't have a Quantum Computer Lab staffed with researchers? Well, Google does. Here they are on G+. To learn what they are up to the Verge has A first look inside Google's futuristic quantum lab. The lab is partnership between NASA, Google, and a 512-qubit D-Wave Two quantum computer.
One result from the lab is:
The first practical application has been on Google Glass, as engineers put the quantum chips to work on Glass's blink detector, helping it to better distinguish between intentional winks and involuntary blinks. For engineering reasons, the quantum processor can never be installed in Glass, but together with Google's conventional server centers, it can point the way to a better blink-detecting algorithm. That would allow the Glass processor to detect blinks with better accuracy and using significantly less power. If successful, it could be an important breakthrough for wink-triggered apps, which have struggled with the task so far.
Google thinks quantum computing has a major role in machine learning:
Tuesday, October 29, 2013 at 8:55AM 
If any of these items interest you there's a full description of each sponsor below. Please click to read more...