Network Automation

What is automation ?

Deploying and configuring boxes is a tedious, boring process which is prone to error when performed manually. Imagine facing the task to change a small piece of the configuration in all devices part of a rather big deployment, whether a campus or a datacenter, connecting to each device and manually changing its configuration. Some changes might need specific values depending on the device’s location or type. This process fast becomes prone to error and could lead to important downtime. We are network engineers, great at building things, not at mimicking a manuscript. Seeking to make our life more interesting and eliminating repeatable tasks is our mission. From this, automation was born.  Automation can take many forms: low level custom scripting written in Python, Perl or Ruby, open automation tools, up to off the shelf commercial products like Junos Space or Infoblox NetMRI.

Devops

Programming languages are known to be structural and mature in the way of abstracting complexity and especially good at repeating the same tasks over and over again, each time adjusting the inputs and getting different but consistent outputs. Compilers have matured over the course of many years and the subject of a lot of research to provide deep and proactive syntactical and logic checking allowing developers to find potential issues early in the process.
By using those development tools into the automation of repeatable infrastructure management tasks lead to the word DevOps, short for Development and Operations. DevOps gives engineers the tool to describe the expected state of their infrastructure in a well-defined language, allowing the use of classes, inheritance, polymorphism and macros to create a structured definition of the state at any point in time. Software development also resulted in a range of tools for change management, process flow, auditing and accountability. In DevOps, pretty much the same tools can be leveraged without change – like using version control systems (e.g. Git, SVN, Mercury) to track changes or using them to create a process flow and let junior engineers submit changes which requires senior engineers to approve those before they are actually applied on the infrastructure.

Automation vs Orchestration ?

Automation and orchestration are often mentioned in the same sentence. Although both operate close to each other, there is a clear distinction between them. Automation is about automating repeatable operations tasks while orchestration is about managing the dynamic nature of an infrastructure. While the result of automation is a persistent change in the configuration, orchestration works with ephemeral (volatile) state. Orchestration will for example allow a dynamic network infrastructure to adapt to movements of virtual machines from one physical host to another (e.g. by changing the state in VTEPs for VXLAN) while automation on the other hand can help to assure that whenever a new VM is created, the corresponding VLANs are configured on the trunk connected to the physical host.
As often the case, there is a grey area. Take the Junos Space Network Director virtual/physical mapping feature. Space ND will not only automate the provisioning of VLANs on switch ports as described previously, but allows up to a certain point to “dynamically” move the VLAN definition between physical ports whenever virtual machines are moved between physical hosts. Space ND uses configuration changes and commits them to make this happen – this is a form of orchestration (the dynamics) but it could be labeled as automation as their results are persistent in nature.

Who needs automation ?

Anyone who needs to manage several devices individually will gain from automation. Which tools are better fit will depend on the number of devices, the knowledge of the team and what features commercial or open off-the-shelf tools can offer. Consider a smaller data center with 10 top of rack switches ; VCF would typically be a good fit while Junos Space Network Director provides configuration management, monitoring, analytics and for those situations where there is a need to customize or automate certain things the Space Platform’s templates and configlets can be leveraged.
On the other side of the spectrum, a mega scale data center (MSDC) with 1000s of ToRs deployed in a L3 IP spine and leaf topology where each device is managed individually and where the compute infrastructure teams have experience with open automation tools like Ansible, Puppet or Chef, the obvious choice would be to follow the infrastructure team’s choice.

Who drives the choice for automation ?

DevOps originates from the server infrastructure people. Originally it was used to keep usernames, passwords … (files basically) and software packages in sync across a large number of unix based machines (physical and/or virtual). The use of these tools for other infrastructure types, like networking, came later and by consequence it is natural that, typically, the choice for a certain automation tool is led by the server infrastructure people.

Which are the most popular automation tools ?

Ansible is an open-source software platform for configuring and managing computer configurations. It combines multi-node software deployments, ad hoc task execution, and configuration management. Ansible manages nodes over SSH and does not require additional remote software to be installed on the devices. It is written in Python and uses Playbooks written in YAML to express reusable descriptions of systems. Some examples of users include Spotify, Twitter, Atlassian (Atlassian which you might know as the people behind Bitbucket and Confluence)

Puppet is another open-source configuration management utility written in Ruby. The user describes system resources and their state in “Puppet Manifests”. The Puppet agent on each device discovers system information through a utility called “Facter” while the Puppet Master compiles the Puppet manifests into a system-specific catalog containing resources and resource dependency which can be matched against the facts and any required changes applied against target devices. Any actions taken by the Puppet agent on the device are then reported back to the Puppet Master. Examples of users of Puppet are PayPal, Yandex, and Rackspace.

Chef is yet another open configuration management tool written in Ruby and Erlang. It uses a pure Ruby, domain specific language for writing system configuration “Recipes”. These recipes describe a series of resources that should be in a particular state: e.g. packages that should be installed, services that should be running, or files that should be written. Chef makes sure each resource is properly configured and corrects any resources that are not in the desired state. Users of Chef are Yahoo!, Facebook, and Klarna.

Which Juniper devices support Puppet, Chef and Ansible ?

Puppet for Junos is available as an add-on package for a select number of devices and software releases. In its most current release, version 1.0 dated March 2014, the package is supported on:

• EX4200, EX4500, EX4550 – Junos 12.3R2
• MX5-80 – Junos 12.3R2
• MX240-960 – Junos 12.3R2
• QFX3500, QFX3600 – Junos 12.3X50-D20.1
• QFX5100 – Junos 13.2X51-D15 with enhanced automation (later OS releases also supported)

More information and downloads are available from the Puppet for Junos landing page: http://www.juniper.net/techpubs/en_US/release-independent/junos-puppet/information-products/pathway-pages/index.html

Chef for Junos is currently only available on the QFX5100 running Junos 13.2X51-D15 with automation enhancements. More information available here: http://www.juniper.net/techpubs/en_US/release-independent/junos-chef/information-products/pathway-pages/index.html

Ansible for Junos 1.0 was recently released. Documentation is available from the website: http://www.juniper.net/techpubs/en_US/release-independent/junos-ansible/information-products/pathway-pages/index.html

It has been tested on the following devices running Junos OS:

• EX4200 standalone and as members of a VC
• QFX5100
• MX80
• MX240, MX480 standalone and as members of a VC
• M120
• SRX5000 standalone and as nodes in a chassis cluster

Remark that the modules do use Netconf and therefore should run similarly on all devices running Junos, there is no need to install any client software or have Python running on the devices.

 

The following modules are included in Ansible for Junos OS Release 1.0:

• junos_get_facts — Retrieve device-specific information from the host.
• junos_install_config — Modify the configuration of a device running Junos OS.
• junos_install_os — Install a Junos OS software package.
• junos_shutdown — Shut down or reboot a device running Junos OS.
• junos_zeroize — Remove all configuration information on the Routing Engines and reset all key values on a device.

QFX5100 Switch Automation Enhancements
To maintain the highest quality standards and assure the stability of the Junos software platform, only software packages and executables that are digitally signed by Juniper can be installed and run. Any automation tool which requires an agent to be deployed to the device will require some sort of base software through a signed package from Juniper engineering. For Puppet this would be the Ruby interpreter and the Puppet Agent. Once the Puppet Agent is available on the platform, Puppet will automatically download and install the required modules.
For the QFX5100 there is a special software bundle ‘jinstall-qfx-5-flex-x.tgz’ which is identical to the standard QFX5100 Junos software but with Veriexec disabled. This Junos version allows unsigned programs to be executed. This version keeps some safeguards to ensure that essential Junos OS files cannot be overwritten, but reserves a 1GB user partition to store unsigned binaries and software packages. Python, Chef and Puppet are included in the image. The switch automation enhancements change some default behavior like a L3 configuration as the factory default compared to an L2 with the standard image for Zero Touch Provisioning (ZTP).
More information about the QFX5100 Switch Automation Enhancements available here: http://www.juniper.net/documentation/en_US/junos13.2/topics/concept/junos-flex-overview.html

Junos scripting: SLAX and JUICE
The configuration and all operational command output in the Junos operating system is based on XML. Changes to this configuration or output can be performed using XML transformations. The CLI is an example of a client that applies transformations to the configuration XML. XML transformations are performed through XSLT (eXtensible Style Sheet Language Transformations) based on the instructions provided in a XSL (eXstensible Stylesheet Language) definition. Scripting on Junos is done by creating XSL files (scripts) that take as input the original configuration and which commits changes as a new XML configuration that is the result of the XSL applied to the original XML: New XML = XSL( Old XML ). XSL is a cumbersome language to write in and why Juniper engineers worked on an alternative language based on constructs from Perl and C. This alternative language is called SLAX or Stylesheet Language Alternative syntaX.SLAX allows you to write scripts that transform the configuration or that output information, along the way taking input from various sources like the command line, operational commands, external sources, local files. SLAX scripting makes extensive use of XML document parsing and formatting and the Xpath query language.

Junos provides 3 types of scripts:
• Commit scripts are executed during commit time and can provide additional processing and influence the outcome of the commit.
• Event scripts are triggered by system events like link down, SNMP trap, …
• Operational scripts that are executed at the operational prompt using ‘op <scriptname> args….’ command.

The Junos SLAX script interpreter library has been made available as an open source project and been used to create a SLAX development environment for unix based systems called JUICE. JUICE allows you to run unaltered on-box Junos SLAX scripts on an off-box server. JUICE can be used for offloading the Junos control plane from CPU intensive scripts, for developing, debugging and testing scripts before staging them to a Junos device or to use the power of SLAX to create new tools like ‘jsnap’. Jsnap is a community SLAX script for JUICE that provides device configuration and runtime snapshots and allows validation of configuration changes through inspecting the state before and after the change. It provides a language to write checks and detect problems in large configurations based on the output of operational commands.

More information about SLAX and JUICE:
https://techwiki.juniper.net/Automation_Scripting/010_Getting_Started_and_Reference/SLAX
Information and the scripts for jsnap can be found at:
https://github.com/Juniper/junos-snapshot-administrator

NETCONF
The Network Configuration Protocol is a network management protocol developed and standardized by the IETF. Netconf provides mechanisms to install, manipulate, and delete the configuration of network devices. Its operations are realized on top of a simple remote procedure call (RPC) layer. The NETCONF protocol uses XML based data encoding for the configuration data as well as the protocol messages. The protocol messages are exchanged on top of a secure transport protocol like SSH or SSL.
Netconf is the language used by Junos Space to remotely manage and monitor Junos devices. Netconf can also be used from any programming language to create custom automation tools. Github.com/Juniper hosts several libraries for programming and scripting languages including Python, Perl, Golang, Ruby, Java and PHP. Using the libraries is optional, any programming language that can process XML and connect SSH or sockets can be used for automating configuration changes and monitoring Junos devices.

Junos Space Templates and Configlets
The Junos Space platform has some built-in tools that can help you to automate deployments and configurations up to a certain extend. Junos Space provides XML based configuration templates which can be instantiated and where placeholders (variables) can be populated during deployment. It allows one to keep pieces of Junos configuration in a central place and deploy them to multiple and different devices. A typical example would be to separate the configuration of DNS and NTP servers in a template which is common for all devices managed through Space. Whenever DNS or NTP changes occur, updating a single template and redeploying it to all devices is all that is required to implement the change across the infrastructure.
Configlets are configuration tools provided by Space that enable the user to quickly apply configurations to devices. A configlet is a configuration template written in the Velocity Template Language (VTL). The configlet will be transformed to a CLI configuration string by the Apache Velocity Template engine before being applied to a device. The dynamic elements (strings) in configuration templates are defined using variables. These variables act as an input to the process of transformation, to construct the CLI configuration string. These variables can contain anything: the interface name, device name, description text, or any similar dynamic values. The values of these variables are either defined by the user or the system, or determined by the context at the time of execution.

Junos Space REST API
The Junos Platform provides a REST API that allows automation of tasks that typically can be performed through the Space GUI. Using HTTP POST requests and XML or JSON encoded responses it is possible to create scripts in any language that interacts with Junos Space. It is possible for example to write a script that automatically provisions a device in Space after it is deployed using ZTP (meaning the stand-alone ZTP, Space Network Director also provides a framework for facilitating a ZTP setup and through SNMP traps it can automatically import newly deployed devices into Space).
Another example is the use of the REST API to leverage the Junos Space platform to automate configuration changes to all devices. Consider creating a script that directly connects to all the devices individually. This script would need some way of authentication and manage certificates or passwords for each connection. Using the Space REST API it is possible to perform remote procedure calls (RPC) directly to the managed devices and piggy-back onto the Space DMI connection. Space provides the security and management of the credentials and your script will take advantage of features like job management, auditing, configuration change management and version control.

ND-API
The Network Director API provides a set of software orchestration services that are exposed through REST APIs. The REST APIs enable network management functions, including:
• Virtualization of cloud and datacenter operations
• Provisioning of secure multitenant networks in a shared network infrastructure
• Support for Layer 2, Layer 3, security, and Internet services
• Support for a single point of integration with external cloud and datacenter orchestration tools
ND-API provides a higher abstraction level than the low level device interface REST API discussed above. This higher abstraction makes it possible to more easily and faster create tools that need to provision multitenant networks in a shared infrastructure. Examples of Juniper tools that leverage this ND-API are the OpenStack Neutron plugin and the CloudStack Network Guru plugin.
Note that ND-API is not included with the Network Director Application but provided as a separate Space app that needs to be installed in addition to ND.

Junos Fusion
Junos Fusion grew out of the Junos Node Unifier (JNU). Junos Node Unifier (JNU) coupled routers and switches in “port extender” and “feature rich” modes. Fusion is the evolution of JNU and brings together a cluster of elements into a single logical entity managed from a supervisor node. Provisioned, managed and configured through the Netconf protocol, it simplifies the network — effectively grouping elements into one, easier to manage, node.
Junos Node Unifier was built entirely in SLAX scripting – at one moment it consisted of over 100,000 lines of code, proving that SLAX is not just a nice to have but an actual tool for automating complex management tasks. Fusion is the integration of JNU as native code in the Junos operating system.
The initial phase of Fusion between Juniper MX and EX products should be visible via CLI in Junos 14.1. In 15.1 router-port extender will become a native Junos feature and support for third party systems via Yang are planned post 15.1.
Fusion ships with Junos and is an integral port of the operating system, no additional licenses required.

JNU 1.3 documentation is available here: http://www.juniper.net/techpubs/en_US/jnu1.3/information-products/topic-collections/design-guide-jnu/junos-node-unifier1.3J1.pdf