One platform — standardized, built and operated by many.
One platform — standardized, built and operated by many.
Automating network infrastructure is essential for improving efficiency, scalability, and accuracy, particularly in large-scale environments. In our setup, we’ve developed a system that automates the provisioning and management of a fleet of SONiC switches by leveraging tools like Zero-Touch Provisioning (ZTP), Open Network Install Environment (ONIE), and NetBox as our single source of truth. This configuration reduces the need for manual setup, minimizes human error, and accelerates deployment by centralizing control within NetBox, ultimately streamlining device provisioning and configuration across the network.
The architecture for automating the provisioning of SONiC switches relies on several key components:
The DHCP server plays a critical role in the initial provisioning phase. When a new SONiC switch connects to the network, the DHCP server assigns it an IP address and provides options for ONIE and ZTP, guiding the switch to download the SONiC image and its configuration.
The DHCP server can employ various configuration strategies, such as using the vendor-class-identifier to filter devices. Some of these strategies are detailed in ZTP docs and ONIE docs.
Below is an example of a basic DHCP configuration for a host in the SCS lab environment:
host <host> {
hardware ethernet <mac-address>;
server-name "<server-name>";
option host-name "<host-name>";
fixed-address <ipv4-address>;
option default-url "<http-server-address>/sonic-image.bin";
option bootfile-name "<http-server-address>/ztp.json";
}
The HTTP server stores essential files, including the SONiC image and configuration scripts, which ZTP uses to set up the switch. This server needs to be accessible from the network where the switches are deployed.
NetBox serves as the backbone of this automated setup, providing a structured and centralized source of truth for device configurations and management scripts. The configuration for SONiC switches combines a configuration template, the device’s state as defined in NetBox, and configuration context with static settings such as IP addresses for NTP servers.
The following diagram shows the flow of information used to generate the final SONiC configuration:
Configuration context NetBox device state
| |
v v
--------------------------------
| SONiC configuration template |
--------------------------------
|
v
------------------------
| SONiC configuration |
------------------------
The SONiC configuration is dynamically generated based on the information stored in NetBox and retrieved via its API. This approach ensures consistency, as the configuration for each switch can be managed centrally and tailored to each device’s specifications.
ONIE is an open-source boot loader designed to automate the installation of a network operating system (NOS) on bare-metal network switches. Developed by Cumulus Networks and now widely supported by the Open Compute Project (OCP), ONIE enables network flexibility by supporting various NOS options, including SONiC, across different hardware platforms.
When a switch with ONIE boots, it enters a discovery mode where it attempts to locate and install a NOS. This process can be configured through multiple methods, with DHCP being one of the most common. In our DHCP configuration, ONIE retrieves a designated NOS image URL, allowing it to automatically download and install the appropriate SONiC version as required.
ZTP is an automated method that streamlines the setup of SONiC network devices, allowing them to be deployed directly to a site, download configuration files, and begin operation without any manual intervention. This approach significantly reduces configuration time, minimizes human error, and simplifies scaling.
In our SCS LAB deployment, ZTP enables the initial configuration of network switches. The ZTP configuration file includes steps to execute essential scripts, retrieve configurations from NetBox (our source of truth), and validate connectivity. Specifically, the provisioning script within this file pulls the switch’s configuration from NetBox based on its hostname, ensuring each device receives the appropriate settings, such as interface configurations, VLANs, BGP configuration and more. Once the configuration is applied, the device automatically reboots, entering its operational state.
The ZTP configuration file we used is structured as follows:
{
"ztp": {
"01-prerequisites-script": {
"plugin": {
"url": "<http-server-address>/prerequisites.sh"
}
},
"02-provisioning-script": {
"plugin": {
"url": "<http-server-address>/provision.sh",
"ignore-section-data": true,
"args": "--netbox-url <netbox-url> --netbox-token <netbox-token>"
}
},
"03-connectivity-check": {
"ping-hosts": [
"<ip-address-that-should-be-reachable>"
]
},
"reboot-on-success": true
}
}
For a detailed example of scripts, visit the related repository.
In modern network infrastructures, managing switch configurations can become a complex task, especially when dealing with scalable environments. To streamline network configuration for SONiC devices, we’ve implemented an MVP (Minimum Viable Product) integration between NetBox and SONiC. This integration leverages NetBox’s data management capabilities with custom scripts that help automate configuration synchronization for SONiC devices, making it easier to keep network configurations consistent and compliant.
NetBox has been enhanced to support configuration management for SONiC devices through two custom scripts that automate
configuration tasks. These scripts SONiC Config Diff and SONiC Config Sync are designed to align SONiC device
configurations with what’s defined in NetBox. Currently, these scripts are executed manually and offer limited
functionality, but they set the groundwork for more comprehensive network management.
+----------------------+
| NetBox |
+----------------------+
/| Sonic Config Diff |\
/ | Sonic Config Sync | \
/ +----------------------+ \
/ | \
/ | \
+-------------------+ +-------------------+ +-------------------+
| SONiC SW 1 | | SONiC SW 2 | ... | SONiC SW N |
+-------------------+ +-------------------+ +-------------------+
This script identifies configuration discrepancies between SONiC devices and NetBox. By comparing the running
configuration on SONiC devices with the stored configuration in NetBox, Config Diff highlights differences that could
affect compliance. This script builds on the netbox-config-diff plugin, extending its capabilities to support SONiC NOS.
When run, SONiC Config Diff produces a report showing where device settings deviate from the intended configuration, allowing administrators to easily spot inconsistencies. These results are presented in a visual format, making it easy to assess compliance at a glance:
While Config Diff identifies differences, SONiC Config Sync enables administrators to apply updates to the SONiC device based on NetBox configurations. This synchronization process allows NetBox to push missing configuration elements to SONiC devices, helping bring them up to date with the desired setup. Config Sync is a step toward automated configuration management, though it only adds configurations and doesn’t support full bidirectional synchronization yet. Like Config Diff, it can be targeted to specific SONiC devices or run across multiple devices using filters like site, rack, or device role:
The result is then visualized as follows:
This blog covers the provisioning and management of SONiC switches,advancing the automation of scalable network environments. By combining NetBox’s management capabilities with the flexible, open-source SONiC platform, we are laying the foundation for simplifying administration and reducing operational complexity.
⚙️ Happy automation!