Func: Fedora Unified Network Controller

I recently started using a tool that I find very handy. It is named func and it is a remote api for management, configuration, and monitoring of systems. What does that mean exactly? I'll get into that but first a little background.

In my day job I manage a number of Linux systems. Some are servers and more are desktop machines in labs used by students. All of the lab machines are triple-boot (Windows XP Pro, CentOS 5.4, and Fedora 12). Fedora has a lot of updates... and it is hard to keep up with them. Typically I have to ssh into each machine to work on it but most of what I do is the same thing over and over again. Wouldn't it be nice to be able to manage multiple machines at once with one command line? That is what func does for you. func allows you to manage remote machines with one command line in parallel.

func was written by Fedora developers mainly to help them manage the server infrastructure that makes up the Fedora distribution's online public servers and build systems. They have an active mailing list that you are encouraged to participate in if you are interested in asking questions and helping to shape the future development of func.

func is written in Python and comes with a number of modules that are custom built for certain tasks. If there is an existing module for your task(s), use the existing module. If not, you can use the command module which basically allows you to run whatever command(s) you want on your remote machines.

func Setup

func operates as a client / server setup. The server is called the master and the clients are called minions. Installing and configuring func is fairly simple and straight forward. Install func on the desired server machine and on all of the clients to be managed.

Server Setup

Setting up the server is as easy insuring the certmaster service is set to run in your default runlevel, starting up the service, making an optional change to the default configuration file, and making sure your firewall isn't blocking the port that certmaster listens on (51235). It's as easy as:

yum install func
chkconfig certmaster on
service certmaster start

You might want to edit the /etc/certmaster/certmaster.conf file and make the following change:

autosign = no

to

autosign = yes

After making that change be sure to restart the certmaster service with:

service certmaster restart

Client Setup

To install and configure func on the client machine just ssh or local login to each client and do the following:

yum install func
chkconfig funcd on

Before starting the funcd service on the client there is one mandatory config change that needs to be done. Make sure and edit /etc/certmaster/minion.conf file and edit the following line:

certmaster = certmaster

to

certmaster = {master hostname}

Replace {master hostname} with the FQDN hostname of the server. Now you can start func:

service func start

The funcd service listens on port 15234 so make sure to open that in your client firewall if it is blocked.

Using func

To see that the certmaster / funcd - client / server is working just look in the /var/lib/certmaster/certmaster/certs/ directory for .cert files. There should be a cert file for each client. func uses certificates between the server and the client and all communications are encrypted with SSL.

You can also do the following as root from the server to verify that your minions are active and listening.

func "*" ping

That will return a minion list and show the status of each client.

Here are some example func commands to give you an idea of some of its capabilities:

func "cslab*" call yumcmd update

That will run a yum update on any machines that have a hostname that starts with cslab.

func "cslab1.*" call command run "df -h" 

That will run df -h on any machine that matches the given wildcard. df -h was just an example. You can run whatever command you want by putting it in quotes. If you want to run multiple commands, just separate them with a ; like you normally would on the command line.

func host1.mydomain call system list_modules

That will give you a list of func modules available on the client host1.mydomain.

I haven't had time to investigate many of the func modules and have mainly been using the command module to run arbitrary commands. The func command does have quite a bit of help built in so check that out as well as the func man page.

Drawbacks

One thing that I don't like about func is the fact that if you address a minion that isn't available (it isn't turned on or in another OS for example), it has a timeout and can slow down the responses and final completion of a func command. If you have a number of unavailable minions the accumulated timeout values can be annoyingly long. It isn't a bad idea to do a func ping, see which machines are available, and then address only those machines to avoid addressing unavailable machines.

Conclusion

I've just scratched the surface with what func can do and I recommend if you have any use for such a utility, install it ASAP and give it a test drive. I have found it to be very handy. Check out the func website for additional documentation.

Python programmers can also integrate func into their programs or write additional modules to better streamline specific use cases. General admin types like me are mostly happy with the ability to manage multiple machines with one command line.