Friday
May082009
Publish/subscribe model does not scale?
Friday, May 8, 2009 at 4:49AM
on Wiki someone posted
"...For relatively small installations, pub/sub provides the opportunity for better scalability than traditional client-server, through parallel operation, message caching, tree-based or network-based routing, etc. However, as systems scale up to become datacenters with thousands of servers sharing the pub/sub infrastructure, this benefit is often lost; in fact, scalability for pub/sub products under high load in large deployments is very much a research challenge."
Does anyone have something to say regarding scaling Publish/subscribe models?
"...For relatively small installations, pub/sub provides the opportunity for better scalability than traditional client-server, through parallel operation, message caching, tree-based or network-based routing, etc. However, as systems scale up to become datacenters with thousands of servers sharing the pub/sub infrastructure, this benefit is often lost; in fact, scalability for pub/sub products under high load in large deployments is very much a research challenge."
Does anyone have something to say regarding scaling Publish/subscribe models?
Reader Comments (2)
It's all in how it's implemented. There's no reason pub/sub can't be scaled, it's just one of those problems that evolves drastically at different levels of scale and is hard to do correctly while giving good service and low latency. For example, in AWS you could imagine a SQS queue for each event-client pair and that would scale just fine. Ramming events into a single server with a million clients not so much.
One option is using a distributed broker network e.g. NaradaBrokering, Gryphon, Oracle Advanced Queuing,
TIBCO Rendezvous, IBM WebSphere MQ, and Padres.
Second option is to use a Load balancer with intelligent routing. e.g. http://srinathsview.blogspot.com/2011/06/scaling-up-your-publishsubscribe.html