Letting the Genie out of the bottle

Imagine if you could transform that unstructured Cisco show command screen output into something more useful than just CLI output.

What if there was a way to transform an SSH CLI show command’s response into a RESTful API-like HTTP GET request response in RESTful API-like structured JSON?

Sounds amazing right? Well with Genie this is totally possible. I mentioned the CTAS Genie / pyATS / xPresso solution in My Toolkit post. I also suggested that gathering facts with Ansible is the best place to start with network automation and infrastructure as code.

But the Ansible facts, while impressive, rich, plentiful, and extremely useful, they do not necessarily contain all of the state information that IOS / NXOS CLI show commands provide. Some information, such as CDP neighbors, interfaces, IP addresses, is available with only the ios_facts / nxos_facts modules but for things like the configured Virtual Route Forwarders (vrf) on a router, the IP Address Resolution Protocol (ARP) tables, or the OSPF routing tables you are stuck with crappy old Cisco CLI output right?

Wrong. You now have a magical Genie on your side who is here to grant all your state capture and transformation wishes! And you get more than 3!

TL;DR

– The historic restrictions of using Cisco IOS / NXOS show commands as CLI-only, raw screen / putty logged output, have been lifted.
– Genie parsers provide REST API HTTP GET-like responses to common CLI show commands.
– Ansible integrated allowing for running and parsing show commands at scale.
– I like to create RAW JSON, Nice JSON, Nice YAML, CSV, Markdown, and interactive HTML mind maps from any JSON I can get my hands on. Now I can do it with Cisco show commands!
– Fill the gaps from what is missing from base Ansible facts.
– Build a powerful, dynamic, state aware documentation set for every device in your network from every day show commands.
– Not as difficult as you might think.
– Another modern network automation, infrastructure as code, tool network engineers should include in their skillset.
– The best development team in the business. The Genie / pyATS / xPresso developers have personally helped me out. Find them on Cisco WebEx Teams for often real-time responses.

What is Genie?

Genie is a parser that automatically converts Cisco IOS / NXOS command output into structured JSON. This structured JSON allows developers to then work more programmatically with the output from important, but otherwise useless, show command output.

For example I am using Genie to parse some key show commands and create a dynamic automated library of documentation in different formats.

You can also go a step further with pyATS and write boolean tests (true / false) in Python using the Genie parsed data as your source of test data. The show ip ospf neighbor command, for example, can be executed, parsed with Genie, and then tested with pyATS! All of this can then be wrapped in business logic, scheduling, and protected with RBAC in xPresso.

Amazing but I am not made of money – how much does all this capability cost?

It is all free.

How do I integrate it with Ansible?

The amazing Ansible integration that I am using is thanks to Clay Curtis and his incredible contributions. Thanks to Clay there are two installation steps on top of the standard Ansible installation and an open Python filter plugin – then you are ready to go.

Please visit the Ansible Galaxy role, Cisco DevNet Code Exchange, and GitHub repository for all the details.

Show and Tell

It’s easier to just demonstrate how the Parser can be used with Ansible. Some prerequisites:

– Linux host
– pip install ansible
– pip install genie
– ansible-galaxy install clay584.parse_genie
– SSH access to network devices from this host
– Credentials for the device (prompted)
– The parse_genie Python filter_plugin
– Make sure your ansible.cfg specifies the location of the parse_genie file in filter_plugins.

[defaults]
filter_plugins=../filter_plugins

Example: show vrf

Recall what a show vrf looks like at the CLI:

This could spawn for pages depending on how many VRFs are hosted on the router. Also – how do you work with this output? Log to a putty file and inspect in notepad offline? Not very user friendly.

Let’s Genie parse that same command and see what the output looks like as structured JSON and take the Pepsi Challenge against the CLI.

In a playbook called CiscoVRFFacts.yml I first scope the playbook (CampusDistribution), then prompt for username and password. Note the collection includes Clay’s clay584.genie collection.

Next I run my standard Cisco show command with the ios_command module and register the response (which is RAW unparsed IOS config at this point) Nothing fancy here.

The next step is where we use the filter_plug in to parse the registered raw response and register a new variable that holds the parsed output. Again – this is not very complicated once you understand the syntax.

Note the parsed command is the same as the original command, in this case show vrf, and we have to specify the operating system (Cisco IOS).

You can optionally print these parsed facts, the nice JSON, to the screen.

Resulting in something like this:

We can save this output to a variety of files and manipulate the output in a variety of ways previously unavailable to us with raw standard CLI IOS output.

For starters lets put the RAW JSON response into a JSON file with the Ansible copy module.

Which is good for forensics, audits, security purposes, or for downstream systems that intake raw JSON, but it’s not very human readable.

Add the Ansible filter | to_nice_json to get the output to look like the output that is printed to the screen.

Now this is up for debate but personally I prefer and find YAML even more human-readable than JSON. Let’s make a YAML file with the | to_nice_yaml filter.

As a reminder this is what the show vrf command looks like at the CLI:

This image has an empty alt attribute; its file name is image-53.png

Now, in YAML:

Incredible!

Going a step further we can try to manipulate the output for business suitable reports in CSV, markdown, and HTML files.

Using yet another Ansible filter, the dict2items, which as the name implies transforms a dictionary to a list of items, we can loop over the original variable {{ pyats_all_vrfs.vrfs }} key and create our CSV / markdown.

(There are some Regular Expression (RegEx) steps that clean up the JSON a bit omitted for brevity)

Add a header row.

And now you have a CSV file!

Similar steps can create a markdown.

And then an HTML mind map can be generated.

Look at all the business and operational value we’ve squeezed out of a simple show vrf command!

All of this code is available on Automate Your Network’s GitHub.

Example: show ip arp

Start with the CLI show ip arp command output, which to be fair isn’t the worst CLI output around, which provides the ARP table for the Global Routing Table.

With more or less the same steps transform this into the same reports.

Setup the playbook:

Run the show ip arp command:

Parse it:

Create your 3 base RAW JSON / Nice JSON / Nice YAML files:

Check out this nice output!

Now anybody, CCNA level or not, can read the ordered structured list and see that VLAN20 has 1 neighbor with an IP of 172.24.2.1, the age, and the MAC address.

Similar steps to transform the output create the CSV / markdown / mind maps:

Also available on GitHub.

Example: show ip arp vrf {{ vrf }}

The exact same steps can be performs by simply adding show ip arp vrf <vrf name> with the same output as the Global Routing Table.

As a development aside I had big plans for show ip arp vrf {{ vrf }} to be a dynamic and automatically loop over all of the VRFs present on the route. I got pretty far but the parser itself hung me up.

Meaning I had a previous loop over the Genie parsed show vrf command which provided me the VRF name to feed the show ip arp vrf command. This all worked out and I could get the raw unparsed list like this:

ios_command:
commands:
– show ip arp vrf “{{ item.key }}”
loop: “{{ pyats_all_vrfs.vrf | dict2items }}”

But when it came time to parse this the following didn’t work.

| parse_genie(command=’show ip arp vrf {{ item.key }}’, os=’ios’)

I think because the parser is treating {{ item.key }} as raw text / the raw command and is not aware of the outer loop and to treat it like a variable. For the same reason I couldn’t write it to prompt for a VRF name. So, yes, I found one edge case drawback where I have to hardcode the VRF. Leave me a note if you see a way around this.

Summary

Genie parsers allow network engineers and operators to transform simple Cisco show commands into REST API-like requests with corresponding structured JSON response from the switch. This all magically happens behind the scenes allowing developers to actually make use of the show command output.

The days of setting up your Putty session to log your console to text files for offline analysis of raw standard output are finally over. Now you can work with JSON which in turn can be transformed into YAML, CSV, markdown, and HTML easily.

Along with Ansible facts, Genie parsed state data can solve the lifelong challenge of creating and maintaining good documentation.

Bonus code – show etherchannel summary

I couldn’t help myself and wrote another GitHub repository after finishing the blog. That’s how much I love Genie and how quick and easy it is!

4 Replies to “Letting the Genie out of the bottle”

    1. Thanks Sean – I should really move towards the newer standardized module. Cisco and Twitter have pointed me at this as well. I will take the time to refactor and maybe make a post with the ‘old’ (this) way and the new (better) way to parse!

Leave a Reply

Your email address will not be published.