Tuesday
Nov202007
Product: SmartFrog a Distributed Configuration and Deployment Framework
Tuesday, November 20, 2007 at 1:05AM From Wikipedia:
SmartFrog is an open-source software framework, written in Java, that manages the configuration, deployment and coordination of a software system broken into components. These components may be distributed across several network hosts. The configuration of components is described using a domain-specific language, whose syntax resembles that of Java. It is a prototype-based object-oriented language, and may thus be compared to Self. The framework is used internally in a variety of HP products. Also, it is being used by HP Labs partners like CERN.
Reader Comments (2)
Todd,
Thank you for the reference! We keep an eye on your site because we like to see how other people deal with the problem of very-big-services and the deadlines that go with them.
SmarFrog is one of the class of "CM systems that work by pushing your system into a declared state", and is not a centrally managed database, but instead a farm of peers that talk over RMI. Because as we all know, a centrally managed database is just a SPOF unless you host it on multiple nodes, and then you are left with the challenges of failover.
One of the interesting things that comes in a distribution is the http://smartfrog.svn.sourceforge.net/viewvc/*checkout*/smartfrog/trunk/core/components/anubis/doc/HPL-2005-72.pdf"> Anubis component. This implements a tuple space that uses multicasts to track who is in the t-space. When a machine exits (i.e. the network has partitioned or someone has just gone ec2-terminate-instances on an instance id), everything gets told. Same if something joins. you can use it as a way of sharing information across nodes: facts you publish "I am the resource manager" get shared for all to see. You can use it for queuing work too: publish work to do into the space, when you pick up a job, declare that too. If the network partitions, everyone can see if someone who had accepted a job has disappeared, and mark it up for re-execution. This is how we partition frame rendering workload in projects like http://www.dshed.net/SE3D/">SE3D, where there are 30 customers buying CPU time on a 400 server farm.
SmartFrog is best at configuring/managing Java code, because it can get very intimate with the code to run. Where it is more limited is in base system configuration, because Java is deliberately kept at arms length from the unix and windows sysadmin API calls. You also have to have java up and running before you can put SmartFrog on there. We have RPMs, we have SSH support, and we have tools that can take a list of machines and install the RPMs there. The other issue is long-haul connectivity. RMI doesn't cut it. We've done stuff in WS-* world, as part of Grid related standards, but that is a different kind of problem -and you still cant send notifications through firewalls. Right now I'm thinking of something RESTy with a web based GUI that can run locally or remotely. I'm also planning on using XMPP as the communications channel. We have some prototypes of bits of this up in sourceforge. Active participation is always welcome!
-Steve Loughran, HP Labs
Thanks for adding your experiences Steve! I am constantly amazed at all the stuff out there that I don't know about.