Dynamic Loops in Ansible

I’ve done many great things with Ansible but occasionally I come across a logical problem that may stretch the tool past it’s limitations. If you have been following the site I am on a big facts discovery and automated documentation movement right now using Ansible Facts and Genie parsers.

The latest parser I am trying to convert to documentation is the show access-lists command.

So we use the Ansible ios_command module and issue the command then parse the response. This is a playbook called CiscoACLFacts.yml

I always start with creating Nice JSON and Nice YAML from the structured JSON returned by the parsed command:

Then I examine the Nice JSON to identify “how” I can parse the parsed data into business documentation in the form of CSV, markdown, and HTML files.

And I have my CLI command converted into amazing structured human or machine readable JSON:

So here is where the logical challenge comes into play with Ansible.

As you know an Access Control List (ACL) is a list (hint: it’s right in the name) of Access Control Entries (ACEs) which are also a list. Both the ACL and the ACE are variable – they could be almost anything – and I need 2 loops: an outer loop to iterate over the ACLs (shown above is the “1”) and an inner loop to iterate over the ACEs (shown above as the “10”).

So I brush up on my Ansible loops. Looks like I need with_nested.

Problem: Ansible does not support dynamic loops as you would expect. Here is what I “wanted to do” and tried for a while before I figured out it wasn’t supported:

with_nested:
– “{{ pyats_all_access_lists | dict2items }}”
– “{{ item[0].value.aces }}”

No go. “Item not found” errors.

Here is the work around / solution:

Before I get into the loops a couple things to point out to anyone new to infrastructure to code or JSON specifically. The Genie parsed return data is not a list by default. Meaning it cannot be iterated over with a loop. We have to filter this from a dictionary – as indicated in JSON by the { } delimiters – into a list (which would be indicated by [ ] delimiters in the JSON if it was a list we could iterate over) – before we can loop over it.

| dict2items is this filter.

The loops:

You can define the outer loop key using loop_var as part of loop_control along with include to build a dynamic outer / inner loop connection.

In order to create my CSV file:

1 – Delete the file outside the loops / last Ansible task before entering the loops

2 – * Important new step here *

We need to perform the outer loop, register the key for this outloop, and then include a separate YAML file that includes the inner loop task

3 – * Another important new step here *

Create the referenced file CiscoACLInnerLoop.yml with the inner loop task, in this case, the task to add the rows of data to the CSV file

Things to identify in the above task:

The loop – it is using the outer loop (the loop_var) ACL_list as the primary key then we turn the .value.aces dictionary into another list with | dict2items giving us the inner list we can iterate over.

Important – the inner loop is what Ansible will reference from this point forward meaning item. now references the inner items. In order to reference the outer key you need to reference the loop_var again as seen on the line: “{{ ACL_list.key }},{{ item.key }}

This gives us the ACL then the individual ACE per row in the CSV file! Mixing the outer and inner loops!

Recommendation – you will notice the start of an {% if %} {% else %} {% endif %} statement – because almost everything in an ACL and ACE list is variable you should test if each item.value.X is defined first, use the value if its defined, otherwise use a hard coded value. As such:

{% if item.value.logging is defined %}

{{ item.value.logging }}

{% else %}

No Logging

{% end if %}

Next, back in the main playbook file, outside the loop, we finally add our CSV header row:

For the sake of keeping this short there is likely some Regular Expression replacements we need to make to clean up any stray JSON or to remove unnecessary characters / strings left behind but in essence we have the follow show access-lists command rendered into CSV:

Operations, management, compliance and standard, and most of all IT SECURITY is going to love this! All of this is in a central Git repository so all of these artifacts are Git-tracked / enabled. All of the CSV files are searchable, sortable, filterable, and EXCEL ready for the business!

Summary

Before you give up on any problem make sure you find and read the documentation!

I have to revisit some previous use cases and problems now with fresh eyes and new capabilities because I had given up on transforming some previous dictionaries in dictionaries because I didn’t know what I was doing!

One step closer! I hope this article helped show you how dynamic Ansible looping is done and you don’t have to fail and struggle with the concept like I did. I am out there on Twitter if you have any questions!