Design of a Modern Cache

This is a guest post by Benjamin Manes, who did engineery things for Google and is now doing engineery things for a new load documentation startup, LoadDocs.

Caching is a common approach for improving performance, yet most implementations use strictly classical techniques. In this article we will explore the modern methods used by Caffeine, an open-source Java caching library, that yield high hit rates and excellent concurrency. These ideas can be translated to your favorite language and hopefully some readers will be inspired to do just that.

Eviction Policy

A cache’s eviction policy tries to predict which entries are most likely to be used again in the near future, thereby maximizing the hit ratio. The Least Recently Used (LRU) policy is perhaps the most popular due to its simplicity, good runtime performance, and a decent hit rate in common workloads. Its ability to predict the future is limited to the history of the entries residing in the cache, preferring to give the last access the highest priority by guessing that it is the most likely to be reused again soon...

Click to read more ...

The Pattern Bible for Distributed Computing

Software design patterns are an emerging tool for guiding and documenting system design. Patterns usually describe software abstractions used by advanced designers and programmers in their software. Patterns can provide guidance for designing highly scalable distributed systems. Let's see how! Patterns are in essence solutions to problems. Most of them are expressed in a format called Alexandrian form which draws on constructs used by Christopher Alexander. There are variants but most look like this:

• The pattern name
• The problem the pattern is trying to solve
• Context
• Solution
• Examples
• Design rationale: This tells where the pattern came from, why it works, and why experts use it

Patterns rarely stand alone. Each pattern works on a context, and transforms the system in that context to produce a new system in a new context. New problems arise in the new system and context, and the next ‘‘layer’’ of patterns can be applied. A pattern language is a structured collection of such patterns that build on each other to transform needs and constraints into an architecture. The latest POSA book Pattern-Oriented Software Architecture Volume 4: A Pattern Language for Distributed Computing will guide the readers through the best practices and introduce them to key areas of building distributed software systems using patterns. The book pulls together 114 patterns and shows how to use them in the context of distributed software architectures. Although somewhat theoretical it is still a great resource for practicing distributed-systems architects. It is as close as you're going to get to a one-stop "encyclopedia" of patterns relevant to distributed computing. However it is not a true encyclopedia since "over 150" patterns are referenced but not described in POSA Volume 4. The book does not go into the details of the pattern's implementations, so the reader should already be familiar with the patterns, or be prepared to spend some time researching. The pattern language for distributed computing includes patterns such as:

• Broker
• Client-Dispatcher-Server
• Pipes and Filters
• Leaders/Followers
• Reactor
• Proactor

Patterns can indeed be useful in designing highly scalable systems and solving various problems related to concurrency, synchronization and resource management and other topics. Wikipedia has more details on Pattern languages to check out.

Click to read more ...

Design Preparations for Scaling

Hi there, what do you think is crucial in the code designing of a scalable site? How does one prepare for webfarms and clusters (e.g. in PHP)? Thanks, Stephan

Click to read more ...