Intro to Redis Cluster Sharding – Advantages, Limitations, Deploying & Client Connections

Redis Cluster is the native sharding implementation available within Redis that allows you to automatically distribute your data across multiple nodes without having to rely on external tools and utilities. At ScaleGrid, we recently added support for Redis Clusters on our platform through our fully managed Redis hosting plans. In this post, we’re going to introduce you to the advanced Redis Cluster sharding opportunities, discuss its advantages and limitations, when you should deploy, and how to connect to your Redis Cluster.

Sharding with Redis Cluster

Click to read more ...

You Can Now Store All Your Stuff on Your Own Google Like File System

New update: Parascale’s CTO on what’s different about Parascale. Let's say you have gigglebytes of data to store and you aren't sure you want to use a CDN. Amazon's S3 doesn't excite you. And you aren't quite ready to join the grid nation. You want to keep it all in house. Wouldn't it be nice to have something like the Google File System you could use to create a unified file system out of all your disks sitting on all your nodes? According to Robin Harris, a.k.a StorageMojo (a great blog BTW), you can now have your own GFS: Parascale launches Google-like storage software. Parascale calls their softwate a Virtual Storage Network (VSN). It "aggregates disks across commodity Linux x86 servers to deliver petabyte-scale file storage. With features such as automated, transparent file replication and file migration, Parascale eliminates storage hotspots and delivers massive read/write bandwidth." Why should you care? I don't know about you, but the "storage problem" is one the most frustrating parts of building websites. There's never a good answer that is affordable. Should you build a SAN or a NAS? How do you make it redundant? How do you make it perform? How do you back it up? How do you grow it without a defense appropriations sized budget? Should you use RAID? Which level and where for what reason? Should you use SCSI, iSCSI, SAS, SATA, or alpha beta? Which vendor should you use? There are so many conflicting opinions about everything. It's all a confusing mess to me. So I like the simplicity of buying commodity nodes with just a bunch of disks attached. But the question has always been how do you turn all those disks into a unified storage system without writing a ton of software on top? Harris says this is what Parascale has done for you:

VSN, like GFS, builds availability and scalability around low-cost servers and disks. NAS appliances rely on costly low-volume boxes that are closed and don't scale. GFS has been deployed in production clusters of over 5,000 servers, proving the scalability of the architecture. Fast, reliable, low-cost and massively scalable storage powers the growth of new applications like Web 2.0, video-on-demand, and hi-resolution image archiving. Parascale is the first of a new generation of software-only storage solutions.

They make a big deal out of it being a software only system. Harris says why this is a good thing:

I like software-based systems because hardware is a commodity. When you create custom hardware you also create low-volume, high-cost components whose economics go from bad to worse. If you *need* to do it, then go for it. But data is getting cooler and the requirement for specialized high-performance hardware is shrinking relative to the market.

Other systems use an appliance model. Appliances can add a lot of value, but they are also a way of monetizing you. A software system on commodity hardware has the potential to give good value. Will it? I didn't see pricing so it's hard to tell. Even odder is their pricing model. You are leasing the software per year, per disk spindle. Do you have any idea how much this will cost? Neither do I. I sounds like it could be horribly expensive or really reasonable. We'll have to see. Another thing that bothers me is that you can't run a database on top of their file system. This means I need an entire separate storage system for my database. You can run a database on a NAS or SAN, so this is a definite disadvantage. Anyway, it's just another interesting option to consider when architecting your website.

Related Articles

LiveJournal created an open source distributed file system called MogileFS that builders may find useful.

Parascale Announces Industry's First Software-Only Storage Solution for Digital Content

Click to read more ...

Paper: The Clustered Storage Revolution

If the clustered file system, clustered storage system, storage virtualization movement is new to you then this is a good intro paper. I's a both vendor puff piece and informative, so it might be worth your time.

A Quick Hit of What's Inside

Clustered storage architectures have the ability to pull together two or more storage devices to behave as a single entity. Clustered storage can be broken down into three types: * 2-way simple failover clustering * Namespace aggregation * Clustered storage with a distributed file systems (DFS)

Click to read more ...

Isilon Clustred Storage System

The Isilon IQ family of clustered storage systems was designed from the ground up to meet the needs of data-intensive enterprises and high-performance computing environments. By combining Isilon's OneFS® operating system software with the latest advances in industry-standard hardware, Isilon delivers modular, pay-as-you-grow, enterprise-class clustered storage systems. OneFS, with TrueScale™ technology, powers the industry's first and only storage system that enables linear or independent scaling of performance and capacity. This new flexible and tunable system, featuring a robust suite of clustered storage software applications, provides customers with an "out of the box" solution that is fully optimized for the widest range of applications and workflow needs. * Scales from 4 TB ti 1 PB * Throughput of up to 10 GB per seond * Linear scaling * Easy to manage

Related Articles

Inside Skinny On Isilon by StorageMojo

Click to read more ...