Cisco Facts – A Collection of Ansible IOS / NXOS Facts and Genie Parsing Playbooks

Cisco DevNet Code Exchange has published my repository !

Dark Mode

Cisco_Facts (this link opens in a new window) by automateyournetwork (this link opens in a new window)

Ansible playbooks that use the IOS / NXOS Facts modules and Genie parsed commands to transform RAW JSON into business-ready documentation

Here you can easily start capturing Ansible Facts for IOS and NXOS and transform the JSON into CSV and Markdown !

Also included are a bunch of valuable Genie parsed show commands which transform the response into JSON then again transforms the JSON into CSV and Markdown!

The playbooks use Prompts so you should be able to clone the repo and update the hosts file and start targeting your hosts! For full enterprise support I suggest you refactor the group_vars and remove the prompts moving to full Ansible Vault – but for portability and ease of start-up I’ve made them prompted playbooks for now.

I would love to hear how they work out for you – please comment below if you have success!

Cisco Services APIs Ansible Playbooks – Version 2.0

I am very pleased to release the Cisco Services API Ansible Playbooks Version 2.0 which has been approved and released on Cisco DevNet Code Exchange !

You can find the code here and here

This major revision basically shifts away from lineinfile to Jinaj2 Templates for scale, performance, readability, and general best practices.

Serial 2 Info

The Cisco Serial 2 Info API receives a valid serial number and then returns structured JSON with your Cisco Contractual information !

The playbook uses the Genie parser to parse the show inventory command

After authenticating against the OAuth 2 service to get a Bearer token

It provides the API the serial number for every part per device.

The API provides the following information back:

Which we first dump into JSON and YAML files

Then template into CSV and MD

Using Jinja2

Which gives us:

Recommended Release

The other, very similar, Ansible playbook uses the Cisco Recommended Release API to create a spreadsheet with the current image on a host and the Cisco recommended version for that host given the Part ID (PID)

Here we don’t even have to use Genie to parse we can use the Ansible Facts module

And we transform again with Jinja2

And get this create report!

Please reach out to me directly if you need any help implementing these playbooks but I believe the instructions and code to be easy enough any beginner, with a little bit of refactoring and thought, could use this code as a starting point in their automation journey.

Dark Mode

Cisco_API_v2 (this link opens in a new window) by automateyournetwork (this link opens in a new window)

Ansible playbooks that capture serial number and PID and send them to the Cisco.com APIs transforming the response into business-ready documents. Version 2.0 uses Jinja2 templates.

A Recipe For Success – Using Ansible to Transform Cisco Data Centre Facts into Business-Ready Documentation

One of my favourite recipes is the Hakuna Frittata both because not only am I a big fan of puns, I also enjoy this hearty vegetarian meal that even I can handle putting together.

Inspired by this simple recipe I have decided to try and document my highly successful Ansible Cisco NXOS Facts playbook that captures and transforms raw facts from the data centre into business-ready documentation – automatically.

Ansible Cisco NXOS Facts to Business-Ready Documentation
Prep: 60-90 Min
Cook: 2-3 Min
Serves: An entire enterprise

Ingredients

1 Preheated Visual Studio Code
1 Git repository and Git
1 stick of Linux (a host with Ansible installed and SSH connectivity to the network devices)
3 pinches of Python filters
1 Cup of Ansible playbook (a YAML file with the serially executed tasks Ansible will perform)
1 Cup of Ansible module – NXOS_Facts
2 Tablespoons of Jinja2 Template
1 Teaspoon of hosts file
1 Tablespoon of group_vars
2 Raw Eggs – Cisco NXOS 7000 Aggregation Switches

Helpful Tip

This is not magic but did not necessarily come easy to me. You can use debug and print msg to yourself at the CLI. At each step that I register or have data inside a new variable I like to print it to the screen (one to see what the data, in JSON format, looks like; and two, to confirm my variable is not empty!)

Directions

1. You will need to first setup a hosts file listing your targeted hosts. I like to have a hierarchy as such:

hosts
[DC:children]
DCAgg
DCAccess

[DCAgg]
N7K01
N7K02

[DCAccess]
N5KA01
N5KB01
N5KA02
N5KB02

Or whatever your logical topology resembles.

2. Next we need to be able to securely connect to the devices. Create a group_vars folder and inside create a file that matches your hosts group name – in this case DC.yml

DC.yml
+

3. Create all the various output folder structure you require to store the files the playbook creates. I like something hierarchical again:

4. Create a playbooks folder to store the YAML file format Ansible playbook and a file called CiscoDCAggFacts.yml

In this playbook, which runs serially, we first capture the facts then transform them into business-ready documentation.

First we scope our targeted hosts (hosts: DCAgg)

Then we use the NXOS_Facts module to go gather all of the data. I want all the data so I choose gather_subset : – all but I could pick a smaller subset of facts to collect.

Next, and this is an important step, we take the captured data, now stored in the magic Ansible variable – {{ ansible_facts }} and put that into output files.

Using the | to_nice_json and | to_nice_yaml Python filters we can make the “RAW JSON” inside the variable (one long string if you were to look at it) into human-readable documentation.

4b. Repeatable step

NXOS Facts provides facts that can be put into the following distinct reports:

Platform information (hostname, serial number, license, software version, disk and memory information)
A list of all of the installed Modules hosted on the platform
A list of all IP addresses hosted on the platform
A list of all VLANs hosted on the platform
A list of all of the enabled Features on the platform
A list of all of the Interfaces, physical and virtual, including Fabric Extenders (FEX)
A list of all connected Neighbors
Fan information
Power Supply information

For some of these files, if the JSON data is structured in way that lends itself, I will create both a Comma-Separated Values (csv; a spreadsheet) file and a markdown (md; “html-light”) file. Some of the reports is just the csv file (IPs, Features, VLANs specifically).

The follow code can be copied 9 times and adjusted by updating the references – the task name, the template name, and the output file name – otherwise the basic structure is repeatable.

In order to create the HTML mind map you will also need mark map installed.

Another example of the code – this is the Interfaces section – notice only the name, src, and dest file names need to be updated as well as the MD and HTML file names in the shell command.

5. The Jinja2 Templates

Now that we have finished our Ansible playbook we need to create the Jinja2 templates we reference in the Ansible template module (in the src line)

Create the following folder structure to store the templates:

roles\dc\dc_agg\templates

Then, for each of the 9 templating tasks, create a matching .j2 file – for example the “base facts” as I like to call them – CiscoDCAggFacts.j2

In this template we need an If Else End If structure to test if we are templating csv or markdown then some For Loops to iterate over the JSON lists and key value pairs.

Add a header row with columns for the various fields of data. Reference your Nice JSON file to find the key value pairs.

No “For Loop” is required here just straight data from the JSON

Since its not csv it must be md; so add the appropriate markdown header rows

Then add the data row using markdown pipes for delimiters instead of commas

Close out the If

An example with For Loops might be Interfaces or Neighbors but the rest of the syntax and structure is the same

Now because there are multiple interfaces I need to loop or iterate over each interface.

Now add the row of data

Note you can include “In-line” If statements to check if a variable is defined. Some interfaces might not have a Description for example. Test if it is defined first, and if not (else) use a default of “No Description”

Other fields are imperative and do not need to be tested.

Close the Loop

Now do the markdown headers for Interfaces

Then the For Loop again and data row again but using pipes

Then close out the If statement

Complete the remaining templates. Save everything and Git commit / push up to your repo.

Cooking Time

Lets run the playbook against two fully-loaded production Nexus 7000s using the Linux time command

Two minutes in the oven !

Results

Some samples of the output.

First the Nice JSON – note the lists have been collapsed to be brief but any of the lists can be expanded in VS Code for the details

Interfaces

Neighbors

Now some prefer YAML to JSON so we have the exact same data but in YAML format as well

Now the above is already incredible but I wouldn’t call JSON and YAML files “business-ready” – for that we need a spreadsheet!

The real tasty stuff are the CSV files!

The general facts

Interfaces

Note that you can filter these csv files directly in VS Code – here I have applied a filter on all interfaces without a description

This captures all types of interfaces

Including SVIs

The Markdown provides a quickly rendered VS Code or browser experience

And the Interactive HTML is pretty neat!

Now remember we have all of these file types for all of the various facts these are just a few samples I like to hand out to the audience – for the full blown experience you can hopefully follow this recipe and cook your own Cisco NXOS Ansible Facts playbook!

This image has an empty alt attribute; its file name is image-72.png

Please reach out if you need any additional tips or advice ! I can be reached here or on my social media platforms.

Collaboration is the key to automation success – A Genie Success Story

I’ve been on about a three year journey with network automation and while I have had great personal and technical success – my organization and most of those outside my immediate day-to-day circle are still shackled to the ‘old’ way of doing things (primarily the CLI).

This year I decided to start a new program – a training session for those outside of my small development team – primarily targeted at the “operations” staff who can benefit the most from automation and infrastructure as code. This includes network operators, monitoring / NOC team members, IT Security staff, other developers, compute (server / storage) teams; everybody is welcome. We divided the calendar up into different teams with different recurring timeslots.

In advance I had written and tested a bunch of Code for the Campus Core – Ansible Facts and Genie parsed show commands – transforming the output into business-ready documentation. My plan was simple enough:

1. Ensure everybody was on the same page and had the same toolkit
* VS Code
* Git
* Azure DevOps repository bookmarked
* Various VS Code extensions
* A basic overview of main vs working branches in Git
* A basic outline on Ansible, YAML, Jinja2, and JSON

2. An operator would create a working development branch – in our case the Distribution Layer – so dist_facts Git branch.
* This operator ‘drives’ the whole session sharing their screen
* Step by step, line by line, refactoring (fancy way of saying copy-pasting) working code from the Core and updating it for the Distribution Layer as necessary
* Git clone, add, commit, push, and pull performed in both VS Code and Linux CLI
* Ansible playbook executed with a –limit against one building, then at scale after validating output
* Thorough tour of the JSON, YAML, CSV, MD, and HTML files after each run

3. Work through Ansible Facts and various Genie parsed commands to build up a source of truth

So far it has been a very successful approach with the two teams adopting Marvel superhero teams (TEAM: CAPTAIN AMERICA and TEAM: IRON MAN respectively) allowing me to create memes like this:

Image

Anyway – back to the point – today we had the following exchange:

Me: “So – we just parsed the show etherchannel summary CLI command and transformed the output into CSV files – amazing right! Any questions?”

Operator catching on quickly: “We use the show interfaces trunk command often to track down what VLANs are being carried on which interfaces – can we transform that into a CSV?”

Me, excited and proud of the Operator: “Amazing question and I’m glad you brought up a practical example of a command you use in the field all the time we can maybe transform into something a little more useful than CLI output!

Launch the Genie parser search engine (under available parsers on the left menu) and let’s see if there is a parser available

Bingo! Let’s do it!”

The playbook

In this example we are targeting the Campus Core.

The playbook is simple enough

Use the Ansible ios_command module to issue and register the show interfaces trunk command

Next, using the Genie filter plugin

Filter the raw variable and register a new variable with the Genie parsed results

Note you need to pass the parse_genie filter two arguments the command itself and the appropriate Cisco operating system

Next I like to create a Nice JSON and Nice YAML file with the parsed results as follows:

Which look like this:

And this:

I then use a loop to create a CSV and MD file from a Jinja2 template

The Jinaj2 Template looks like this

Couple things to hightlight:

* We challenge the current iteration of the Ansible loop and when it is on “csv” we template a CSV file format otherwise (else) it will be “md” and we template a markdown file
* The For Loop is over each interface in the results
* Be careful with the CSV file you need to regex_replace the comma out of results because they are comma-separated which will throw off your CSV file. The markdown does not require any regex.

Which results in this amazing sortable, searchable, filterable, version controller, source controller, truthful, fact-based CSV file:

Now this example is just the singular logical Core but we will quickly refactor the code next week in our next session together and the operator who wanted this playbook will get a chance to write it ! Then we will have the interface trunk information for the entire campus automatically in spreadsheets!

The moral of this story is to collaborate. Ask your front-line operators how automation can help them. Do they have any frequently used or highly valuable CLI Commands they want transformed into CSV format? Would Ansible facts help them? And then show them how to do it so they can start writing these playbooks for themselves.

It’s always DNS – but it doesn’t have to be!

DDI Automation – A collaborative live demonstration with BlueCat

I had the pleasure of collaborating with my friends Dana Iskoldski and Chris Meyer over at BlueCat exploring and automating DNS, DHCP, and IP Address Management (IPAM) (DDI) with the BlueCat Address Manager (BAM) and the BlueCat Gateway.

Infrastructure as Code and network automation is not always about switches and routers – there is incredible value to be found automating all the way up the stack into the critical Layer 7 services like DNS and DHCP; or transforming from Microsoft Excel-based IP address management to a fully automated solution.

In this 2-part series watch us explore and ultimately automate DNS using the BlueCat BAM and Gateway APIs with Postman, Ansible, and Python

Part One – BAM API Automation with Postman and Ansible

Part Two – BlueCat Gateway Automation

GitHub repository

Prevent an Intrusion – Run the Recommended Version!

In the wake of some very high profile IT security breaches and state sponsored attacks using compromised software today I wrote some infrastructure as code Ansible playbooks to create some business-ready documentation to help us understand our Cisco software version footprint against what release the vendor recommends. It is very important to run “Safe Harbor” code in the form of the Gold Star release. These releases are as close as it gets to being bug-free, secure, tested, and supported in production environments.

The ‘old-way’ involved getting the Cisco Part ID (PID) or several PIDs and looking up the recommended release on Cisco.com using an ever deepening hierarchy of platforms, operating systems, and PIDs. At scale this is like a day’s worth of work to go gather all of this information and present it in a way the business can understand.

Building on my recent success with the Serial2Info Cisco.com API as well as Ansible Facts I thought this might be another nice use-case for business-centric, non-technical (not routes, IP addresses, mac addresses, etc), extremely important and critical insight.

Use Case

Can I automatically get the PID from a host or group of hosts and provide it to the Cisco.com Software Suggestion API building business-ready reports in CSV and markdown?

Answer: Yes!

The Playbook

Again you are going to need:

* A Linux Host with SSH access to your Cisco IOS devices and HTTPS access to the Cisco.com API
* Credentials for the host and for the OAuth2 API
* We are not using Genie parsers here so just “base” Ansible will work

Step 1. Setup credential handling

Create a playbook file called CiscoCoreRecommendedReleaseFacts.yml

Again I use prompted methodology here same as the Serial2Info API

Gather the username, enable secret, Cisco.com API ClientID, Client Secret

Step 2. Gather Ansible Facts

Using the ios_facts module gather just the hardware subset

Because we are using Ansible Facts we do not need to register anything – the JSON is stored in the Ansible magic variable ansible_facts

I need 2 keys from this JSON – the PID and ideally the current running version. These can be found as follows in the ansible_facts variable:

Which is accessed as ansible_facts.net_model

Which again is accessed as ansible_facts.net_version

With the information above – without going any further – I could already build a nice report about what platforms and running versions there are!

But let’s go a step further and find out what Cisco recommends I should be running!

Step 2. Get your OAuth2 token

First, using the Ansible URI module

We need to get our token using the registered prompted credentials.

The API requires the following headers and body formatting; register the response as a variable (token):

We have to break apart the RAW JSON token to pass it to the ultimate Recommended Release API:

Now we are ready to send PIDs to the API.

Step 3 – Send PID to Cisco.com API

Again using the URI module:

Here we pass the ansible_facts.net_model Fact to the API as an HTTP GET:

The headers and body requirements. Notice the authentication and how we pass the Bearer Token along. We also register the returned JSON:

Here is what the returned JSON looks like:

The highest level key is json or accessed via RecommendedRelease.json

There is a productlist

Which as you can see is a list as denoted by the [ ]

Inside this list is another product key with the values from the API about the product itself

A little further down we find the recommended software release

Step 4 – Transform technical documentation into business ready CSV / MD files

These JSON and YAML (I also use the | to_nice_yaml filter to create a YAML file along with the JSON file) files are create for technical purposes but we can do a bit better making the information more palatable using business formats like CSV and mark down.

It is just a matter of using Jinja2 to template the CSV and Markdown files from the structured JSON variables / key-value pairs.

Add a final task in the Ansible playbook that will loop over the CSV and MD file types using the template module to source a new .j2 file – CiscoCoreRecommendedReleaseFacts.j2 – where our logic will go to generate our artifacts.

The Jinja2 starts with an If Else EndIf statement that checks if the Ansible loop is on CSV or not. If it is it uses the CSV section of templated file format otherwise it uses markdown syntax.

First we want to add a CSV header row

Then we need a For Loop to loop over each product in the productList

Now we add our “data line” per product in the loop using the various keys

Hostname for example uses the Ansible magic variable inventory_hostname

Then we want the Base PID. We use the Ansible default filter to set a default value in case the variable happens to be empty.

We continue accessing our keys and then we close the loop.

Now we need to create the Markdown syntax

And the same logic for the “data row” but with pipes instead of commas. Make sure to close off the If statement

Step 5 – Run playbook and check results

We run the playbook as ansible-playbook CiscoCoreRecommendedReleaseFacts.yml

Answer the prompts

Let the playbook run and check the results!

Summary

Again with a few free, simple tools like Ansible and the Cisco.com API we can, at scale gather and report on the current running version and the vendor recommended version quickly and easily and fully automatically!

Now go and start protecting your enterprise network armed with these facts!

Sign on the Dotted Line

Layer 9 issues – finance – are often some of the most challenging a network engineer faces. Contract management can be particularly difficult in any scale organization especially if you are not “sole source” purchasing. Serial numbers and contracts are also not typically things the “network people” want to deal with but when that P1 hits and you try to open a SEV 1 TAC CASE – only to find out you are not under contract – I’ve been in less terrifying car accidents than this nightmare scenario.

I have good news ! Using a mix of automation and developer-like tools the network engineer can now create a real source of truth that, along with routes and MAC address-tables and other technical information, can include inventory and contractual business documentation from stateful, truthful, real-time, facts from Cisco.

Ok so let’s get into it!

As a rough outline for our logic here is the use case:

Can I automatically gather the serial numbers from Cisco device hostnames and then provide them to Cisco and get my contractual state for each part on each device?

Answer: YES !

What you will need:

* Linux host with Ansible, Genie parser
* Linux host requires both SSH access to the Cisco host and Internet Access to the OAuth2 and Cisco.com API HTTPS URLs
* Cisco SmartNet Total Care – I have written up instructions in this repo under the “OnBoarding Process” section

The Playbook

Step 1 – We will need to get the serial number for every part for a given hostname. For this we will use the standard show inventory command for IOS using the Ansible ios_command module. I will be using prompted methods for demonstration purposes or for on-demand multi-user (each with their own accounts) runtime, but we could easily Ansible Vault these credentials for fully hands-free run time or to containerize this playbook. I am also targeting a specific host – the Core – but I could easily change this to be every IOS device in the enterprise. This playbook is called CiscoCoreSerial2InfoFacts.yml

First prompt for username, enable secret, Cisco Customer ID, Cisco Customer Secret and register these variables:

Then run the ios_command show inventory and register the results in a variable.

Step 2 – Parse the raw output from the IOS command

Next, we use Genie to parse the raw results and register a new variable with the structured JSON. Genie requires, for show inventory, the command, the operating system, and the platform (in this case a Cisco 6500)

And here is what that structured JSON looks like:

So now we have a nice list of each part and their serial number we can feed the Cisco.com API to get back our contract information.

Step 3 – Get an OAuth 2 token from Cisco web services.

Cisco.com APIs use OAuth2 for authentication meaning you can not go directly against the API with a username and password. First you must retrieve a Bearer Token and then use that limited time token within it’s lifetime against the ultimate API.

Using the Ansible URI module go get a token and register the results as a variable. Provide the Customer ID and Client secret prompts to the API for authentication. This is an HTTP POST method.

With the new raw token setup the token type and access token from the raw response

Step 4 – Provide token to the Serial2Contract Cisco API to get back contractual information for each serial number.

Now that we have a valid token we can authenticate and authorize against the Cisco SmartNet Total Care Serial Number 2 Contract Information API.

In this step we are going to use an Ansible loop to loop over the Genie parsed structured JSON from the show inventory command providing the sn key for each item in the list. We need to use the Python | dict2items Ansible filter to transform the dictionary into a list we can iterate over.

The loop is written as

loop: “{{ pyats_inventory.index | dict2items }}”

And each serial number is referenced in the URL each iteration through the loop:

url: https://api.cisco.com/sn2info/v2/coverage/summary/serial_numbers/{{ item.value.sn }}

We register the returned structured JSON from the API as Serial2Info which looks like this:

So now I have the JSON – let’s make it a business ready artifact – a CSV file / spreadsheet and a markdown file – using Jinja2

Step 5 – Using Jinja2 lets template the structured JSON into a CSV file for the business.

Create a matching Jinja2 template called CiscoCoreSerial2InfoFacts.j2 and add a task to Ansible that uses the template module to build both a CSV file and a markdown file from the JSON.

In the Jinja2 file we need a section for CSV (if item = “csv”) and a section for markdown (else) based on their respective syntax. Then we need to loop over each of the responses.

result in Serial2Info[‘results’] is the loop used. I also add a default value using a filter | default (‘N/A’) in case the value is not defined. SFPs for example do not have all of the fields that a supervisor module has so to be safe it’s best to build in a default value for each variable.

The final Jinja2 looks something like this:

Which results in a CSV and Markdown file with a row for every serial number and their contractual facts from the API.

Summary

Large scale inventory and contract information can easily be automated into CSV spreadsheets that the business can easily consume. Ansible, Genie, Cisco.com APIs, Jinja2 templating and a little bit a logic come together into an automation pipeline that ensures contractual compliance and inventory fidelity at scale!

Recognition from the Business

2020 has been an extremely challenging year that required a lot of digital innovation for us at the House of Commons.

The Speaker of the House of Commons, the Clerk, and Members of Parliament recognize my boss, CIO Stephan Aube, and his teams efforts in this clip from ParlVu (a cloud-based service I helped implement)

“The IT department under Mr. Aubé has been very visionary and has been planning to accommodate whatever the Members of the House really want to do…I wanted to compliment them and recognize their hard work and diligence in making sure everything is there for us when we decide as a House.”  – Honourable Anthony Rota

Full video