Topics for this podcast:
*Continuent extends MySQL replication to Oracle Database
*CFEngine updates server automation software
*Devops moving mainstream
*Neo Technology integrates with Spring
*451 CAOS report from Hadoop World
iTunes or direct download (26:56, 4.6MB)
CFEngine is both the oldest and the newest of the popular tools for automating site administration. Mark Burgess invented it as a free software project in 1993, and years later, as deployments in the field outgrew its original design he gave it a complete rethink and developed the powerful concept of promise theory to make it modular and maintainable. In this guise as version 3, CFEngine stands along with two other pieces of free software, Puppet and Chef, as key parts of enterprise computing. Along the way, Burgess also started a commercial venture, CFEngine AS, that maintains both the open source and proprietary versions of CFEngine.
Diego Zamboni has recently taken the position of Senior Security Advisor at CFEngine AS and is writing a book for O'Reilly on CFEngine 3. I talked to him this week about the recent new release of the open source version (3.2.4) in tandem with a new commercial release of CFEngine 3 Nova (version 2.1.3). Here's are excerpts of what he has written to introduce CFEngine 3.
CFEngine 3 is fine-tuned to the features and design that make it possible to automate very large numbers of systems in a scalable and manageable way. CFEngine 3 is also very lightweight--its binaries normally use less than 30MB of disk space, it requires a single TCP port to communicate among servers and clients, and it has been designed to be very resource-efficient. CFEngine 3 can run on everything from smartphones to supercomputers.
CFEngine 3 is different from many other automation mechanisms in that you do not need to tell it what to do. Instead, you specify the state in which you wish the system to be, and CFEngine 3 will automatically and iteratively decide the actions to take to reach the desired state, or as close to it as possible. Underlying this ability is a powerful theoretical model known as Promise Theory, which was initially developed for CFEngine 3, but which has also found other applications in Computer Science and in other fields such as Economics and Organization.
This allows you to develop building blocks for complex promises that remain readable and manageable because the lower-level components are encapsulated. Each promise represents the desired state of certain parts of the system. At the lowest level, these are some of the things that you can express to CFEngine 3 as desired states:
"Make sure file /foo/bar contains line xyz"
"Make sure user foobar exists/does not exist"
"Make sure process foo is/is not running"
At a higher level of abstraction, you can encapsulate CFEngine 3 operations and express high-level desired states:
"Make sure all web servers have Apache installed"
"Make sure all root accounts have the same, centrally-designated password"
"Make sure parameters EnableDNS and AllowRoot are disabled on all sshd configurations"
And at an even higher level, you can express top-level desired states like these:
"Configure host xyz as a database server"
"Create a new cluster of VMs to use as web servers"
So what's in the new versions? CFEngine 3 Nova includes:
System monitoring extensions, which extend the monitoring capabilities of CFEngine 3 Community (to monitor system state such as CPU load, number of processes and network connections, disk utilization, etc.) to allow for defining custom monitors for any type of information.
Support for manipulating virtual machines on Xen, VMware ESX, and KVM.
Native Windows support.
Flexible searching of reports in a brand new scalable interface that supports thousands of hosts on a single hub.
Improved machine learning and anomaly monitoring for diagnostics and capacity planning. Additional sensors have been added to detect operating system performance and behavioral trends, especially on Linux kernels.
The NoSQL document-oriented database MongoDB, used instead of MySQL for all storage on Nova's Mission Portal.
Generic JSON return values so that users can customize the interface and JQuery framework of the Mission Portal. This allows direct access to data in a way that makes higher levels of scripting more effective.
CFEngine 3 Community also includes a large number of improvements, all of which are in Nova too:
A vastly improved bootstrapping process, which makes it easy to get new CFEngine 3 servers and clients up and running with very little manual configuration.
Support for environments, which are a way of grouping hosts according to arbitrary definitions. This makes it very easy to define, for example, "development," "testing," and "production" environments for CFEngine 3 policies.
The new cf-report command, available in both Community and Nova, which allows extraction of data and generation of reports from the command line. It can produce reports both about the behavior of the current CFEngine 3 environment (policies, hosts, etc.) and about internal information, such as a CFEngine 3 syntax summary.
Many performance and concurrency improvements and bug fixes.
Several new functions and parsing improvements, including and(), not(), and or() functions, to ease writing of complex class expressions.
A new and improved Emacs mode for editing CFEngine 3 policy files.
Velocity Europe, being held Nov. 8-9 in Berlin, will bring together the web operations and performance communities for two days of critical training, best practices, and case studies.So about 4 monts ago there was the crazy idea to start a new FOSS event in Belgium targeted at sysadmins.
What started out as an event for local people to meet local people with some local speakers actually ended up being a small local event with some top international speakers on onfiguration mananagement and system administration mixed with a bunch of good local ones !
I had the honour to open the conference with an extremely short version of the Devops talk I gave earlier last year.. extremely short as I knew that over the course of the weekend the topic would reoccur a lot.
We had the first european talk on Chef, by Joshua Timberman, and we had Puppet talks amongst by Dan Bode from Puppetlabs and CFengine talks , devops was a frequently dropped word,
We had a book raffle where we handed out O'Reilly's .. we had a great free pizza party (got the idea from the saturday pizza event at LCA 2005) , and we had some free beer. Sounds like a good combination for a geeky weekend.
Apart from the regular talks there were plenty of Open Spaces where interesting topics were discussed ... we had spaces on Open Source vs Open Core , strong voices were heard when we discussed what we should do with the Open Core companies that claim to value Open Source , some people think we should actually list the fauxpensource ones somewhere and make sure the world knows about them
We had an awesome configuration management discussion session discussing Chef vs Puppet vs CFengine . And much much more ...
Some people owe me plenty of Sushi as I had to do my MySQL HA talk before their Managing MySQL talk , but other than that .. things just went fine..
Today I caught a tweet from Kara Swisher referencing some exclusive news she posted on Boomtown about VMware’s upcoming deal to buy Zimbra from Yahoo! This is would be VMware’s second acquisition of an open source ISV in under a 
year. In August 2009 VMware acquired open source java vendor SpringSource that not only developed the popular Spring framework but had also acquired open source systems management vendor Hyperic (May 2009) and commercial Apache support vendor, Covalent (January 2009).
According to CNET’s Matt Asay, Yahoo!’s Zimbra business unit is still growing and has an impressive customer base:
Lost in the news of Zimbra’s release of version 6.0 of its collaboration suite is the importance of one very big number: 50 million. That’s how many paid mailboxes Zimbra claims now, a number that puts it within spitting distance of IBM Lotus Notes (approximately 145 million paid mailboxes) and Microsoft Exchange (approximately 175 million paid mailboxes). Whatever the truth to rumors that Zimbra is up for sale, Zimbra is an appreciating asset for Yahoo, not a depreciating one.
I also noticed a couple of months ago that VMware started to re-brand getting rid of their old blue logo and moving to a grey logo sans the “virtualization boxes”. According to this post by VMware CMO Rick Jackson:
Now, as we look at our current offerings based on vSphere, and our vision of delivering the infrastructure for unrestrained cloud computing, the image we are portraying to the market has evolved. In fact, our message embodies the notion of freeing IT from the constraints of physical resources.
Makes you wonder in the long-term where VMware might draw the line...
Does this signal the beginning of a broader VMware open source acquisition strategy? Maybe they will complete their java application stack with a database. Barring Larry Ellison offering to sell MySQL to VMware maybe there are some other opportunities. VMware might benefit from picking up EnterpriseDB or maybe become the patron saint for MySQL fork, MariaDB sponsored by MySQL creator Monty Widenius. Beyond the database there are a number of interesting buying opportunities out there for VMware should they have their pocketbook open. For one there is rPath which can build and update Linux virtual machines and provide automated provisioning of systems taking VMware’s management and deployment capabilities one step further.
Another option would be to get deeper in management by picking up one of the open source configuration management vendors like Reductive Labs that produces Puppet or newly funded cloud configuration rival Opscode and their open source project, Chef. They could even go old school and take a look at CFengine which is similar to Chef and Puppet but supports not only Unix-like systems but Windows too. Alternatively, they could acquire commercial open source vendor, Cloudera that provides support for Hadoop, an open source implementation of MapReduce which is ideally suited for cloud deployment.
I guess that’s enough speculation for today. However, it will be curious to see if the deal goes through and if VMware pays a premium over Yahoo!’s acquisition price of $350 million back in 2007. It could as The VarGuy notes it could trigger a reset for how open source companies are valued.
Related Articles
Technorati Tags: Cfengine, Cloudera, Covalent, Hyperic, MapReduce, MariaDB, Open Source, Opscode, Puppet, Reductive Labs, Spring Source, VMware, Zimbra
Technorati Tags: 