For integration between ansible and consul I’ve been using a third party python client called consulate. It is decent, however both it and consul are new and it doesn’t support the full consul HTTP API yet.

Currently I’m trying to model our topology in consul’s key value store but lists of values are not intuitive. Consul seems to only store strings, so without doing string parsing / casting I am unable to get complicated values out.

For example, I’d like to store something like this:

/roles/zookeeper/zones = [‘us-west-2a’, ‘us-west-2b’, ‘us-west-2c’]

and when I do a query against consul I could get back a list to work with. Since I can’t I had been doing terrible things like this:

def get_keys(self):
        output = {}
        for k, v in self.session.kv.items().iteritems():
            if not v is None and type(v) is str and v.startswith(”[”) and v.endswith(“]”):
                output[k] = v.replace(’[’, “).replace(‘]’, “).split(‘, ‘)
            else:
                output[k] = v
        return output

Reading the Consul API docs closer though I found that they have a weird implementation to support this. Specifically:

It is possible to also only list keys without their values by using the “?keys” query parameter along with a GET request. This will return a list of the keys under the given prefix. The optional “?separator=” can be used to list only up to a given separator.

For example, listing “/web/” with a “/” seperator may return:

[ “/web/bar”, “/web/foo”, “/web/subdir/” ] Using the key listing method may be suitable when you do not need the values or flags, or want to implement a key-space explorer.

So I dug into the (simple) consulate find method and added support for ?keys to it. The pull request logged with consulate and we can use my consulate fork for the time being.

I’ve been building out automation for deploying micro services in ec2. We’re using consul for service registration, discovery, health checks, and configuration. Since consul provides an available key value store for configuration we’ve been trying to define the topology that way. Ansible has some very good documentation and it is one of the things I like most about the project.

Documentation for building your own dynamic inventory is fairly complete, but I was having trouble with including variables in that inventory. The dynamic inventory documentation shows examples of of host level variables in it’s json output:

eg:

{    n“databases”: {        n“hosts”   : [ “host1.example.com”, “host2.example.com” ],        n“vars”    : {            n“a”   : true        n}    n}n}

However, in the standard inventory documentation there is a differnt type of variable, specifically, group variables.

[atlanta]nhost1nhost2

[atlanta:vars]nntp_server=ntp.atlanta.example.comnproxy=proxy.atlanta.example.comnnSo my assumption was I could use the group variable syntax in the dynamic inventory output to achive the same thing, the power here was that different consul instances could contain different values allowing me to build a fairly dynamic infrastructure. Combining the documentation from the static inventory with the dynamic output gave me something that looked like this:

{    n“databases”: {        n“hosts”   : [ “host1.example.com”, “host2.example.com” ],        n“vars”    : {            n“a”   : true        n}    n},n“databases:vars”: {n“postgres”: {n“version”: “9.3”n} n}n}

Unfortunately ansible-playbook wanted nothing to do with this. ‘databases:vars’ was being cast to a list and treated as a group which was stomping the variables I was trying to pass around. I spent a while thinking about the problem and decided that inventory wasn’t actually where I needed these variables passed in. Instead it would be fine to use consul as a facts source and use that to drive role behavior. I started out trying to augment the magic variable ‘ansible_facts’ by modifying the setup module but ultimately I didn’t want to maintain my own core module. Instead I was able to find two good blog posts about writing a fact gathering module. The first is a little old but comes from one of the best ansible blogs I’ve found. Ansible: variables, variables, and more variables explains the basic approach well but the example code seemed to no longer work. Docker: Containers for the Masses — The docker_facts module is a much more recent post and had a working example to go along with it. It turned out the reason the first example wasn’t working was due to how the module was exiting. The first example was just printing the output, now the correct approach is to use the module.exit_json method.

module.exit_json(changed=changed, ansible_facts=ansible_facts)

I’ve posted the module on my ansible fork and may sent a pull request over.