router image

Network Config Templating using Ansible, Part1

By Kirk Byers
2014-02-23

In this article, I will show you how to use Ansible to generate network device configurations based on a template and a variables file. The article presupposes that you have Ansible installed on your system and you have some basic familiarity with Ansible. The Ansible documentation is very good. You can find it at http://docs.ansible.com/. You might want to read through the Getting Started section and through some of the Intro to Playbooks section. Both of these sections are fairly short.

The general problem that we are trying to solve is—we want a systematic way of creating network device configurations based on templates and variables.

For very brief background—Ansible is an open-source automation application that can be used to automate many tasks in your environment (predominantly compute and cloud tasks). Ansible can also generate files based on Jinja2 templates and variables. Jinja2 is a widely-used Python templating system, see http://jinja.pocoo.org/ for more information about Jinja2.

In this example, I am going to generate the configuration files of five remote-office routers (note, in part1 of this series I only generate small configuration snippet files; in part2, I will generate full configuration files). The example presupposes a large amount of common configuration between the devices. Once the templating system is in place, the number of devices does not significantly matter (at least to the range of a hundred devices or so).

In this example, I am only going to build the configuration of one type of device (small remote-office routers). You could expand the configuration system to multiple device types by adding new Ansible roles (for example, access-layer switches, distribution-layer switches, etc.). You could also expand the system by adding additional logic. For example, you could add logic to support different router models.

This article does not discuss how to automatically load the configuration files onto devices. It only discusses the process of creating the configuration files themselves. There are zero-touch provisioning mechanisms available from several network vendors. Additionally, if the devices are connected to the network with basic IP information, you could automate the configuration load through a script.

Enough with the preliminaries—let's get started. I am running Ansible version 1.4.5 on an Amazon EC2 instance. My /etc/ansible/hosts file is just configured to use localhost:

[gituser@ip ~]$ cat /etc/ansible/hosts 
#[local]
localhost ansible_ssh_user=gituser

User 'gituser' will both run the playbook and be the 'remote' SSH user. I put 'remote' in quotes because Ansible is just going to SSH back to localhost. The SSH trust is setup such that no password is required for this SSH to localhost.

Let's start by creating a base directory that will contain the Ansible role. I am going to call this directory 'RTR-TEMPLATE' and it will be located in '/home/gituser/ANSIBLE/RTR-TEMPLATE'. Inside this directory, create the following directory structure:

[gituser@ip RTR-TEMPLATE]$ find . -type d
.
./roles
./roles/router
./roles/router/vars
./roles/router/templates
./roles/router/tasks

This directory structure is used to take advantage of Ansible roles. You can find more details about Ansibile roles at http://docs.ansible.com/playbooks_roles.html.

Starting in ./RTR-TEMPLATE, I create a file called site.yml. site.yml contains the following:

---
- name: Generate router configuration files
  hosts: localhost

  roles:
    - router

The '---' line just indicates that this file is a YAML file. In this file, we could potentially have a series of plays, each play is a set of actions that Ansible could take. In our example, we only have a single play. Our play starts with '- name:'. The name keyword just indicates text that will be displayed when the playbook executes. The next line, 'hosts:', specifies which 'remote' computers we are going to execute this play on. Finally, the play specifies a role, 'router', which is going to provide more details about the actions to take.

Using this role 'router' is going to cause several things to happen. First, Ansible is going to look in ./RTR-TEMPLATE/roles/router/tasks/main.yml for any additional tasks to be added to the play. Second, Ansible is going to look in ./RTR-TEMPLATE/roles/router/vars/main.yml for any variable definitions and add them to the play. Third, when we use a template, Ansible will search in ./RTR-TEMPLATE/roles/router/templates/ to try to locate the template. Ansible roles do other things, but these are the items that we care about in this example.

From the above, I know that Ansible is going to add any tasks that I specify in ./RTR-TEMPLATE/roles/router/tasks/main.yml into the play. Consequenty, I change into this directory and create the 'main.yml' file to be the following:

---
- name: Generate configuration files
  template: src=router.j2 dest=/home/gituser/ANSIBLE/CFGS/{{item.hostname}}.txt
  with_items: 
  - { hostname: twb-sf-rtr1 }
  - { hostname: twb-sf-rtr2 }

Once again this is a YAML file and starts with a triple hyphen. The section starting with '- name:' is the beginning of the first and only task in this file. The 'name' line just contains text to be displayed when this task executes. The 'template:' line specifies that we are going to use a template named 'router.j2' and from this template create a file located at 'dest' and named {{item.hostname}}. I will explain {{item.hostname}} shortly. After the template line is 'with_items:'; with_items is an iterator (it acts like a for loop where the variable 'item' will be set to equal one of with_items elements on each iteration). In this case with_items contains two elements:

  - { hostname: twb-sf-rtr1 }
  - { hostname: twb-sf-rtr2 }

On the first iteration through 'with_items', the task will be executed with dest equal to /home/gituser/ANSIBLE/CFGS/twb-sf-rtr1.txt. On the second iteration, dest will be /home/gituser/ANSIBLE/CFGS/twb-sf-rtr2.txt. Note, the elements inside of with_items are dictionaries; so {{item}}.hostname equals to 'twb-sf-rtr1' and 'twb-sf-rtr2', respectively.

Now that the main.yml tasks file is defined, we need to create the template file named 'router.j2'. Remember that for our 'router' role, Ansible will search in ./RTR-TEMPLATE/roles/router/templates/ looking for our template file. Consequently, we change into this directory and create our template file:

no service pad
service tcp-keepalives-in
service tcp-keepalives-out
service timestamps debug datetime msec localtime show-timezone
service timestamps log datetime msec localtime show-timezone
service password-encryption
!
hostname {{item.hostname}}
!
boot-start-marker
boot-end-marker
!
logging buffered 32000
no logging console

Here, I made a simple template file consisting of a configuration snippet so that I can demonstrate the operation of the Ansible playbook

This is all standard stuff except for {{item.hostname}}. Once again, this is the variable passed in from the Ansible task 'with_items' iterator.

Now I have all the pieces in place to generate the configuration snippets. Consequently, let's execute the playbook:

[gituser@ip RTR-TEMPLATE]$ cd /home/gituser/ANSIBLE/RTR-TEMPLATE/
[gituser@ip RTR-TEMPLATE]$ ansible-playbook site.yml

PLAY [Generate router configuration files] ************************************

GATHERING FACTS ***************************************************************
ok: [localhost]

TASK: [router | Generate configuration files] *********************************
changed: [localhost] => (item={'hostname': 'twb-sf-rtr1'})
changed: [localhost] => (item={'hostname': 'twb-sf-rtr2'})

PLAY RECAP ********************************************************************
localhost                  : ok=2    changed=1    unreachable=0    failed=0 

Now if this worked correctly, I should have two configuration snippet files located in /home/gituser/ANSIBLE/CFGS:

[gituser@ip CFGS]$ ls -al
total 16
drwxr-xr-x 2 gituser gituser 4096 Feb 22 00:19 .
drwxr-xr-x 8 gituser gituser 4096 Feb 21 18:34 ..
-rw------- 1 gituser gituser  323 Feb 22 00:16 twb-sf-rtr1.txt
-rw------- 1 gituser gituser  323 Feb 22 00:16 twb-sf-rtr2.txt

There are the two files and the hostname is set correctly in each file:

[gituser@ip CFGS]$ cat twb-sf-rtr1.txt
no service pad
service tcp-keepalives-in
service tcp-keepalives-out
service timestamps debug datetime msec localtime show-timezone
service timestamps log datetime msec localtime show-timezone
service password-encryption
!
hostname twb-sf-rtr1
!
boot-start-marker
boot-end-marker
!
logging buffered 32000
no logging console
[gituser@ip CFGS]$ cat twb-sf-rtr2.txt
no service pad
service tcp-keepalives-in
service tcp-keepalives-out
service timestamps debug datetime msec localtime show-timezone
service timestamps log datetime msec localtime show-timezone
service password-encryption
!
hostname twb-sf-rtr2
!
boot-start-marker
boot-end-marker
!
logging buffered 32000
no logging console

Let's now relocate our variables to an external variables file and add additional routers. Remember, that Ansible will include any variables located in ./RTR-TEMPLATE/roles/router/vars/main.yml in our play. Consequently, we can just define our router variables here:

[gituser@ip RTR-TEMPLATE]$ cd roles/router/vars/
[gituser@ip vars]$ cat main.yml
---
test_routers:
   - { hostname: twb-sf-rtr1 }
   - { hostname: twb-sf-rtr2 }
   - { hostname: twb-la-rtr1 }
   - { hostname: twb-la-rtr2 }
   - { hostname: twb-den-rtr1 }

I have assigned the variable a name, 'test_routers', and added three routers to it. Now back in ./roles/router/tasks/main.yml, I change 'with_items' to use 'test_routers':

---
- name: Generate configuration files
  template: src=router.j2 dest=/home/gituser/ANSIBLE/CFGS/{{item.hostname}}.txt
  with_items: "{{ test_routers }}"

I then execute the playbook again.


[gituser@ip RTR-TEMPLATE]$ ansible-playbook site.yml

PLAY [Generate router configuration files] ************************************

GATHERING FACTS ***************************************************************
ok: [localhost]

TASK: [router | Generate configuration files] *********************************
ok: [localhost] => (item={'hostname': 'twb-sf-rtr1'})
ok: [localhost] => (item={'hostname': 'twb-sf-rtr2'})
changed: [localhost] => (item={'hostname': 'twb-la-rtr1'})
changed: [localhost] => (item={'hostname': 'twb-la-rtr2'})
changed: [localhost] => (item={'hostname': 'twb-den-rtr1'})

PLAY RECAP ********************************************************************
localhost                  : ok=2    changed=1    unreachable=0    failed=0
                        

Looking at this I output, I see that the first two configuration files were unchanged and that the final three configuration files were created. This illustrates an important principle of Ansible (and of other similar systems). Ansible strives to be idempotent—things should only be changed, if they are not in the correct state.

The ./CFGS directory now contains the five configuration files:

[gituser@ip CFGS]$ ls -ltr
total 20
-rw------- 1 gituser gituser 323 Feb 22 00:16 twb-sf-rtr1.txt
-rw------- 1 gituser gituser 323 Feb 22 00:16 twb-sf-rtr2.txt
-rw------- 1 gituser gituser 323 Feb 22 00:29 twb-la-rtr1.txt
-rw------- 1 gituser gituser 323 Feb 22 00:29 twb-la-rtr2.txt
-rw------- 1 gituser gituser 324 Feb 22 00:29 twb-den-rtr1.txt

And the hostname is set correctly in each of the five files:

[gituser@ip CFGS]$ grep hostname *.txt
twb-den-rtr1.txt:hostname twb-den-rtr1
twb-la-rtr1.txt:hostname twb-la-rtr1
twb-la-rtr2.txt:hostname twb-la-rtr2
twb-sf-rtr1.txt:hostname twb-sf-rtr1
twb-sf-rtr2.txt:hostname twb-sf-rtr2


*** Note from 2017-02-15, newer versions of ansible now require the following format with_items: "{{ test_routers }}". Consequently, I updated this article to reflect this requirement.


This completes Part1 of this series of articles—we now have a working system for generating configuration snippet files from an Ansible task file, template file, and variables file. In Part2 of this series, I expand on this example to generate full device configuration using several variables including conditionals. In Part3 of this series, I expand the system to support multiple models, a template hierarchy, and multiple roles.

If you are interested in receiving future content from me, you can join my email-list or you can follow me on Twitter (@kirkbyers).

Kirk Byers
CCIE #6243
Twitter: @kirkbyers

router image

Network Config Templating using Ansible, Part1

By Kirk Byers
2014-02-23

In this article, I will show you how to use Ansible to generate network device configurations based on a template and a variables file. The article presupposes that you have Ansible installed on your system and you have some basic familiarity with Ansible. The Ansible documentation is very good. You can find it at http://docs.ansible.com/. You might want to read through the Getting Started section and through some of the Intro to Playbooks section. Both of these sections are fairly short.

The general problem that we are trying to solve is—we want a systematic way of creating network device configurations based on templates and variables.

For very brief background—Ansible is an open-source automation application that can be used to automate many tasks in your environment (predominantly compute and cloud tasks). Ansible can also generate files based on Jinja2 templates and variables. Jinja2 is a widely-used Python templating system, see http://jinja.pocoo.org/ for more information about Jinja2.

In this example, I am going to generate the configuration files of five remote-office routers (note, in part1 of this series I only generate small configuration snippet files; in part2, I will generate full configuration files). The example presupposes a large amount of common configuration between the devices. Once the templating system is in place, the number of devices does not significantly matter (at least to the range of a hundred devices or so).

In this example, I am only going to build the configuration of one type of device (small remote-office routers). You could expand the configuration system to multiple device types by adding new Ansible roles (for example, access-layer switches, distribution-layer switches, etc.). You could also expand the system by adding additional logic. For example, you could add logic to support different router models.

This article does not discuss how to automatically load the configuration files onto devices. It only discusses the process of creating the configuration files themselves. There are zero-touch provisioning mechanisms available from several network vendors. Additionally, if the devices are connected to the network with basic IP information, you could automate the configuration load through a script.

Enough with the preliminaries—let's get started. I am running Ansible version 1.4.5 on an Amazon EC2 instance. My /etc/ansible/hosts file is just configured to use localhost:

[gituser@ip ~]$ cat /etc/ansible/hosts 
#[local]
localhost ansible_ssh_user=gituser

User 'gituser' will both run the playbook and be the 'remote' SSH user. I put 'remote' in quotes because Ansible is just going to SSH back to localhost. The SSH trust is setup such that no password is required for this SSH to localhost.

Let's start by creating a base directory that will contain the Ansible role. I am going to call this directory 'RTR-TEMPLATE' and it will be located in '/home/gituser/ANSIBLE/RTR-TEMPLATE'. Inside this directory, create the following directory structure:

[gituser@ip RTR-TEMPLATE]$ find . -type d
.
./roles
./roles/router
./roles/router/vars
./roles/router/templates
./roles/router/tasks

This directory structure is used to take advantage of Ansible roles. You can find more details about Ansibile roles at http://docs.ansible.com/playbooks_roles.html.

Starting in ./RTR-TEMPLATE, I create a file called site.yml. site.yml contains the following:

---
- name: Generate router configuration files
  hosts: localhost

  roles:
    - router

The '---' line just indicates that this file is a YAML file. In this file, we could potentially have a series of plays, each play is a set of actions that Ansible could take. In our example, we only have a single play. Our play starts with '- name:'. The name keyword just indicates text that will be displayed when the playbook executes. The next line, 'hosts:', specifies which 'remote' computers we are going to execute this play on. Finally, the play specifies a role, 'router', which is going to provide more details about the actions to take.

Using this role 'router' is going to cause several things to happen. First, Ansible is going to look in ./RTR-TEMPLATE/roles/router/tasks/main.yml for any additional tasks to be added to the play. Second, Ansible is going to look in ./RTR-TEMPLATE/roles/router/vars/main.yml for any variable definitions and add them to the play. Third, when we use a template, Ansible will search in ./RTR-TEMPLATE/roles/router/templates/ to try to locate the template. Ansible roles do other things, but these are the items that we care about in this example.

From the above, I know that Ansible is going to add any tasks that I specify in ./RTR-TEMPLATE/roles/router/tasks/main.yml into the play. Consequenty, I change into this directory and create the 'main.yml' file to be the following:

---
- name: Generate configuration files
  template: src=router.j2 dest=/home/gituser/ANSIBLE/CFGS/{{item.hostname}}.txt
  with_items: 
  - { hostname: twb-sf-rtr1 }
  - { hostname: twb-sf-rtr2 }

Once again this is a YAML file and starts with a triple hyphen. The section starting with '- name:' is the beginning of the first and only task in this file. The 'name' line just contains text to be displayed when this task executes. The 'template:' line specifies that we are going to use a template named 'router.j2' and from this template create a file located at 'dest' and named {{item.hostname}}. I will explain {{item.hostname}} shortly. After the template line is 'with_items:'; with_items is an iterator (it acts like a for loop where the variable 'item' will be set to equal one of with_items elements on each iteration). In this case with_items contains two elements:

  - { hostname: twb-sf-rtr1 }
  - { hostname: twb-sf-rtr2 }

On the first iteration through 'with_items', the task will be executed with dest equal to /home/gituser/ANSIBLE/CFGS/twb-sf-rtr1.txt. On the second iteration, dest will be /home/gituser/ANSIBLE/CFGS/twb-sf-rtr2.txt. Note, the elements inside of with_items are dictionaries; so {{item}}.hostname equals to 'twb-sf-rtr1' and 'twb-sf-rtr2', respectively.

Now that the main.yml tasks file is defined, we need to create the template file named 'router.j2'. Remember that for our 'router' role, Ansible will search in ./RTR-TEMPLATE/roles/router/templates/ looking for our template file. Consequently, we change into this directory and create our template file:

no service pad
service tcp-keepalives-in
service tcp-keepalives-out
service timestamps debug datetime msec localtime show-timezone
service timestamps log datetime msec localtime show-timezone
service password-encryption
!
hostname {{item.hostname}}
!
boot-start-marker
boot-end-marker
!
logging buffered 32000
no logging console

Here, I made a simple template file consisting of a configuration snippet so that I can demonstrate the operation of the Ansible playbook

This is all standard stuff except for {{item.hostname}}. Once again, this is the variable passed in from the Ansible task 'with_items' iterator.

Now I have all the pieces in place to generate the configuration snippets. Consequently, let's execute the playbook:

[gituser@ip RTR-TEMPLATE]$ cd /home/gituser/ANSIBLE/RTR-TEMPLATE/
[gituser@ip RTR-TEMPLATE]$ ansible-playbook site.yml

PLAY [Generate router configuration files] ************************************

GATHERING FACTS ***************************************************************
ok: [localhost]

TASK: [router | Generate configuration files] *********************************
changed: [localhost] => (item={'hostname': 'twb-sf-rtr1'})
changed: [localhost] => (item={'hostname': 'twb-sf-rtr2'})

PLAY RECAP ********************************************************************
localhost                  : ok=2    changed=1    unreachable=0    failed=0 

Now if this worked correctly, I should have two configuration snippet files located in /home/gituser/ANSIBLE/CFGS:

[gituser@ip CFGS]$ ls -al
total 16
drwxr-xr-x 2 gituser gituser 4096 Feb 22 00:19 .
drwxr-xr-x 8 gituser gituser 4096 Feb 21 18:34 ..
-rw------- 1 gituser gituser  323 Feb 22 00:16 twb-sf-rtr1.txt
-rw------- 1 gituser gituser  323 Feb 22 00:16 twb-sf-rtr2.txt

There are the two files and the hostname is set correctly in each file:

[gituser@ip CFGS]$ cat twb-sf-rtr1.txt
no service pad
service tcp-keepalives-in
service tcp-keepalives-out
service timestamps debug datetime msec localtime show-timezone
service timestamps log datetime msec localtime show-timezone
service password-encryption
!
hostname twb-sf-rtr1
!
boot-start-marker
boot-end-marker
!
logging buffered 32000
no logging console
[gituser@ip CFGS]$ cat twb-sf-rtr2.txt
no service pad
service tcp-keepalives-in
service tcp-keepalives-out
service timestamps debug datetime msec localtime show-timezone
service timestamps log datetime msec localtime show-timezone
service password-encryption
!
hostname twb-sf-rtr2
!
boot-start-marker
boot-end-marker
!
logging buffered 32000
no logging console

Let's now relocate our variables to an external variables file and add additional routers. Remember, that Ansible will include any variables located in ./RTR-TEMPLATE/roles/router/vars/main.yml in our play. Consequently, we can just define our router variables here:

[gituser@ip RTR-TEMPLATE]$ cd roles/router/vars/
[gituser@ip vars]$ cat main.yml
---
test_routers:
   - { hostname: twb-sf-rtr1 }
   - { hostname: twb-sf-rtr2 }
   - { hostname: twb-la-rtr1 }
   - { hostname: twb-la-rtr2 }
   - { hostname: twb-den-rtr1 }

I have assigned the variable a name, 'test_routers', and added three routers to it. Now back in ./roles/router/tasks/main.yml, I change 'with_items' to use 'test_routers':

---
- name: Generate configuration files
  template: src=router.j2 dest=/home/gituser/ANSIBLE/CFGS/{{item.hostname}}.txt
  with_items: "{{ test_routers }}"

I then execute the playbook again.


[gituser@ip RTR-TEMPLATE]$ ansible-playbook site.yml

PLAY [Generate router configuration files] ************************************

GATHERING FACTS ***************************************************************
ok: [localhost]

TASK: [router | Generate configuration files] *********************************
ok: [localhost] => (item={'hostname': 'twb-sf-rtr1'})
ok: [localhost] => (item={'hostname': 'twb-sf-rtr2'})
changed: [localhost] => (item={'hostname': 'twb-la-rtr1'})
changed: [localhost] => (item={'hostname': 'twb-la-rtr2'})
changed: [localhost] => (item={'hostname': 'twb-den-rtr1'})

PLAY RECAP ********************************************************************
localhost                  : ok=2    changed=1    unreachable=0    failed=0
                        

Looking at this I output, I see that the first two configuration files were unchanged and that the final three configuration files were created. This illustrates an important principle of Ansible (and of other similar systems). Ansible strives to be idempotent—things should only be changed, if they are not in the correct state.

The ./CFGS directory now contains the five configuration files:

[gituser@ip CFGS]$ ls -ltr
total 20
-rw------- 1 gituser gituser 323 Feb 22 00:16 twb-sf-rtr1.txt
-rw------- 1 gituser gituser 323 Feb 22 00:16 twb-sf-rtr2.txt
-rw------- 1 gituser gituser 323 Feb 22 00:29 twb-la-rtr1.txt
-rw------- 1 gituser gituser 323 Feb 22 00:29 twb-la-rtr2.txt
-rw------- 1 gituser gituser 324 Feb 22 00:29 twb-den-rtr1.txt

And the hostname is set correctly in each of the five files:

[gituser@ip CFGS]$ grep hostname *.txt
twb-den-rtr1.txt:hostname twb-den-rtr1
twb-la-rtr1.txt:hostname twb-la-rtr1
twb-la-rtr2.txt:hostname twb-la-rtr2
twb-sf-rtr1.txt:hostname twb-sf-rtr1
twb-sf-rtr2.txt:hostname twb-sf-rtr2


*** Note from 2017-02-15, newer versions of ansible now require the following format with_items: "{{ test_routers }}". Consequently, I updated this article to reflect this requirement.


This completes Part1 of this series of articles—we now have a working system for generating configuration snippet files from an Ansible task file, template file, and variables file. In Part2 of this series, I expand on this example to generate full device configuration using several variables including conditionals. In Part3 of this series, I expand the system to support multiple models, a template hierarchy, and multiple roles.

If you are interested in receiving future content from me, you can join my email-list or you can follow me on Twitter (@kirkbyers).

Kirk Byers
CCIE #6243
Twitter: @kirkbyers