Product: Puppet the Automated Administration System
Saturday, October 25, 2008 at 3:57AM Update: Digg on their choice and use of Puppet. They chose puppet over cfengine, and bcfg2 because they liked Puppet's resource abstraction layer (RAL), the ability to implement configuration management incrementally, support for bundles, and the overall design philosophy.
Puppet implements a declarative (what not how) configuration language for automating common administration tasks. It's the system every large site writes for themselves and it's already made for you! Ilike was able to "easily" scale from 0 to hundreds of servers using Puppet. I can't believe I've never seen this before. It looks really cool. What is Puppet and how can it help you scale your website operations?
From the Puppet website:
Puppet has been developed to help the sysadmin community move to building and sharing mature tools that avoid the duplication of everyone solving the same problem. It does so in two ways:
* It provides a powerful framework to simplify the majority of the technical tasks that sysadmins need to perform
* The sysadmin work is written as code in Puppet's custom language which is shareable just like any other code.
This means that your work as a sysadmin can get done much faster, because you can have Puppet handle most or all of the details, and you can download code from other sysadmins to help you get done even faster. The majority of Puppet implementations use at least one or two modules developed by someone else, and there are already tens of recipes available in Puppet's CookBook.
This sound good. But does it work in the field? HJK Solutions' Adam Jacob says it does:
Puppet enables us to get a huge jump-start on building automated, scaleable, easy to manage infrastructures for our clients. Using puppet, we:
1. Automate as much of the routine systems administration tasks as possible.
2. Get 10 minute unattended build times from bare metal, most of which is data transfer. Puppet takes it the rest of the way, getting the machines ready to have applications deployed on them. It’s down to two and a half minutes for Xen.
3. Bootstrap our clients production environments while building their development environment. I can’t stress how cool this really is. Because we are expressing the infrastructure at a higher level, when it comes time to deploy your production systems, it’s really a non-event. We just roll out the Puppet Master and an Operating System auto-install environment, and it’s finished.
4. Cross-pollinate between clients with similar architectures. We work with several different shops using Ruby on Rails, all of whom have very similar infrastructure needs. By using Puppet in all of them, when we solve a problem for one client, we’ve effectively solved it for the others. I love being able to tell a client that we solved a problem for them, and all it’s going to cost is the time it takes for us to add the recipe.
Puppet, today, is a tool that is good enough to handle the vast majority of issues encountered in building scalable infrastructures. Even the places where it falls short are almost always just a matter of it being less elegant than it could be, and the entire community is working on making those parts better.
Related Articles
Reader Comments (13)
cfengine is an alternative to puppet (puppet was written as a way to fix cfengine's shortcomings). I had the choice of both, and ended up choosing cfengine because it looked a lot more straightforward than puppet.
Regardless of what you choose, once you get a box under management that way things get a whole lot easier. Bringing on new servers is a snap, at the moment it's 5 commands which does everything including authentication, nfs, ssh keys, files, etc. Rolling out changes is equally trivial.
Sean
Don't worry that you'd never heard of puppet, even Wikipedia knows nothing of it's existence! :)
http://www.callum-macdonald.com/" title="Callum" target="_blank">Callum
Sean: Very interesting. I'm curious as to what you found not so straightforward with puppet as opposed to cfengine ?
Part of the problem with Puppet seeming not so straightforward has been the documentation. Until fairly recently, there was really only one person (Luke Kanies) working on it with occasional small additions by others. Unfortunately, the documentation suffered more than anything else in many cases.
Lately, this has been improving steadily. The community around Puppet is starting to really grow: the #puppet channel on FreeNode is constantly active and there is a lot of activity on the mailing list.
You probably didn't hear about it because it just hasn't been around that long yet, nor had the exposure of many other tools in the same space. This will change over time.
For the record, Puppet wasn't written as a way to explicitly fix cfengine's failures as to provide a next-generation solution in general. There are lots of tools out there with good ideas -- Bcfg2, LCFG, cfengine, Quattor, SmartFrog, etc. -- but none of them had the modularity or transparency that I wanted.
So yes, Puppet is a response to cfengine (I was actually a big contributor to the community for a while), but it's also a response to all of those other tools and all of my experience and ideas.
I agree that it's taken us a while to get the documentation in shape, and as always, it can still use work, but as mentioned, the IRC channel and user lists are helpful and active, and the product and related docs are constantly getting better. For such a young product (less than three years) I've been really impressed with the community that's coalesced around it.
- There is wikipedia coverage of Puppet in http://en.wikipedia.org/wiki/Comparison_of_open_source_configuration_management_software">Comparison of open source configuration management software . I'm sure a page of its own would be welcome.
- Can I also mention SmartFrog, http://smartfrog.org/">http://smartfrog.org/, which has a very similar syntax, but which is java based rather than built in ruby. That has strengths -full access to the java libraries, intimate control of Java programs such as application servers, secure RMI communications, Eclipse integration, and weaknesses: among them RMI and the prerequisites of what has to be installed before Java works. The book Ant in Action has coverage of it.
For all CM tools, there is startup costs: you need to learn something new, you need to invest time up front writing the base templates. Its very much like the effort in learning a build tool like Make, Ant or Rake. Just like some people try and get their IDE to build everything, but real projects know you need to automate the build to run everywhere, projects need to recognise that the entire act of deploying/configuring a big system can be automated, and if you dont automate it, you end up with an epic nightmare bringing up one system, and no way to scale other than cloning vmware images of the machine you have working (at best, copying the VM HDD, at worst, backing up a physical box and restoring it onto a VM). Automating deployment is the next step in software project automation.
Steve Loughran, SmartFrog team at HP Laboratories
In Extremadura we are using Puppet for remote sincronization in the primary and secundary school servers. We have 620 servers with Linex2006 R3, School edition (Debian Etch + custom packages)
It's working fine and no problems at the moment (if well configurated). The only drawbacks it's that sometimes, when you install puppet, the signature between master server and the others doesn't work, so you have to reset it by hand (but only one time for client server until you change the signature).
http://wiki.debian.org/DebianEdu/LinEx2DebianEdu-Roadmap
PD:
From Spain with `"Write in English experience" = 0` :P
The mention about cfengine is good. It's a viable alternative. There is also bcfg2, NetDirector, and several others.
--
Dustin Puryear
Author, "Best Practices for Managing Linux and UNIX Servers"
http://www.puryear-it.com/pubs/linux-unix-best-practices
The complexity of a puppet setup is vastly overstated.
Or rather — getting started really isn't that hard; even months ago when the documentation was a bit sparser than now. I'm using it to manage our server at http://www.yellowbot.com/ and got it in shape so I can go from "bare metal" to "running the application" with a few commands and a 10-15 minute wait for the kickstart install and initial puppet run.
- ask
Puppet is a well devised system for dealing with application package and configuration management. HJK Solutions has some excellent overviews of integrating Puppet into other systems like iLike and Nagios to provide a comprehensive solution, although it has a ways to go to get to the "usable by anyone" stage.
The only downside to the setup is that it isn't cross platform (which I can not blame Luke for, really). That's a factor that sticks it's nasty head into many larger organizations, and a fail point for using Puppet across an environment that has a significant amount of hosted Win32 applications.
It's also worth mentioning http://open.controltier.com/">ControlTier, which does at least make some mentions of cross platform support while having an open-source core. The commercial applications like BladeLogic and OpsWare are just too hidden from folks who haven't already invested into it to be a useful comparison.
Is it compatible with all operating systems?
Puppet supports most major Unixes:
http://reductivelabs.com/trac/puppet/wiki/DownloadingPuppet
What about N1 Service Provisioning from Sun Microsystems ? N1 is XML template based and I'm wondering what are the benefits of using Puppet over N1 or Opsware ? or Is it just marketing ?