Launching CloudEOS in Azure with Terraform

Introduction

Enterprise cloud organizations are orchestrating environments in the cloud.  This can be done with cloud native tools such as AWS CloudFormation or Azure Resource Manager Templates.  However, Terraform is winning enterprise mindshare as a cross-cloud orchestration system, and this post is an example of a simple CloudEOS deployment into Azure using Terraform.

Diagram

Below is the diagram that will be referenced in this post.

Prerequisites

It will be assumed that the reader has familiarity with Terraform and how to setup the Terraform environment.  For basic instructions on setting up a Terraform environment, see https://www.terraform.io to get started. 

Also, the Microsoft Azure CLI will need to be installed on your workstation to be able to create this example.  More can be found on Microsoft’s site with regard to the CLI installation at https://docs.microsoft.com/en-us/cli/azure/install-azure-cli?view=azure-cli-latest  Enterprises may have different ways on how they want users to login to the Azure CLI, so check with your company if you are using a corporate Azure account on the recommended way to use the Azure CLI.  In many cases, it will simply be ‘az login’ at the command prompt.

Terraform uses resources to define item creations, and the resources are contained in something called a provider within Terraform.  The Azure provider is what will be referenced below.  For more information on the Azure provider, reference this url:  https://www.terraform.io/docs/providers/azurerm/index.html

The Terraform script in this article will be fairly simple and will encompass a main.tf file and a bootstrap file for configuring the CloudEOS instance on startup.  To begin, create a folder that you will use for the Terraform project, and create a name and the ‘.tf’ extension.  For this example, the file will be main.tf.

Provider Definition

Open the main.tf file, and begin by defining the Terraform Azure provider.  When running Terraform, the provider will be retrieved when the initialization process starts.

provider "azurerm" {
    skip_provider_registration = true
    features {}
}

Resource Group Definition

Azure uses a resource group to house all the resources for a given Azure solution.  In this resource group, the “westus2” region will be used for the resources, and the resource group will be called ‘TestResourceGroup’. 

resource "azurerm_resource_group" "TestRG" {
    name ="TestResourceGroup"
    location ="westus2"
}

VNet Definition

Next, define the Azure VNet that will be used as well as the cidr block of addresses.  For this example, 10.2.0.0/16 will be used as the main block of addresses for the VNet.  A Name can also be given to the resource in the form of a tag for making it easier to locate from within the Azure console.

A quick note about resources in Terraform:  A resource is defined with the ‘resource’ key word, then the resource name, ‘azurerm_virtual_network’ in this case, then the name to reference the resource in the current file which is ‘Vnet’ for the below example.

resource "azurerm_virtual_network" "Vnet" {
    name = "TestVNET"
    address_space = ["10.2.0.0/16"]
    resource_group_name = azurerm_resource_group.TestRG.name
    location = azurerm_resource_group.TestRG.location
    tags = {
        "Name" = "TestVNET"
    }
}

Subnet Definition

Now define the two subnets for the VNet to be used for the public (10.1.0.0/24) and private (10.1.1.0/24) side of the CloudEOS instance.  This is where you can divide the larger cidr block into more manageable subnets.  In the example below, PublicSubnet1 is used for the public facing side of the CloudEOS instance.  PrivateSubnet1 is used for the host facing side of CloudEOS and will not be able to communicate outside the VNet except through CloudEOS’s private interface that will be defined below.

A quick note about referencing resources in Terraform:  In the example below, the virtual_network_name is needed to correlate the VPC created with the subnet that is being defined.  To reference the above VNet resource, simply follow this format – [resource name].[name of resource you assigned].name.  The same can be done for the resource_group_name.

resource "azurerm_subnet" "PublicSubnet1" {
    name = "Public"
    virtual_network_name = azurerm_virtual_network.Vnet.name
    resource_group_name = azurerm_resource_group.TestRG.name
    address_prefixes = ["10.2.0.0/24"]
}

resource "azurerm_subnet" "PrivateSubnet1" {
    name = "Private"
    virtual_network_name = azurerm_virtual_network.Vnet.name
    resource_group_name = azurerm_resource_group.TestRG.name
    address_prefixes = ["10.2.1.0/24"]
}

Security Groups

Security Group Definition

Security groups in Azure are like firewall rules that maintain the state of the traffic flowing through them.  Two security groups will be created for this topology.  One will be open for the communication between the CloudEOS instance and the host, and the other will allow only SSH and ICMP traffic inbound from the Internet.  The definition of each of the resources is defined below and will be applied to the subnets

resource "azurerm_network_security_group" "OpenSG" {
    name = "OpenSG"
    location = azurerm_resource_group.TestRG.location
    resource_group_name = azurerm_resource_group.TestRG.name

    security_rule {
        name = "Allow_Any_In"
        description = "Allow all traffic in"
        priority = 100
        direction = "Inbound"
        access = "Allow"
        protocol = "*"
        source_port_range = "*"
        destination_port_range = "*"
        source_address_prefix = "*"
        destination_address_prefix = "*"
    }

    tags = {
        "Name" = "OpenSecurityGroup"
    }
}

resource "azurerm_network_security_group" "InternetSG" {
    name = "InternetSG"
    location = azurerm_resource_group.TestRG.location
    resource_group_name = azurerm_resource_group.TestRG.name

    security_rule {
        name = "Allow_SSH"
        description = "Allow SSH access"
        priority = 100
        direction = "Inbound"
        access = "Allow"
        protocol = "Tcp"
        source_port_range = "*"
        destination_port_range = 22
        source_address_prefix = "*"
        destination_address_prefix = "*"
    }

    security_rule {
        name = "Allow_ICMP"
        description = "Allow ICMP access"
        priority = 150
        direction = "Inbound"
        access = "Allow"
        protocol = "Icmp"
        source_port_range = "*"
        destination_port_range = "*"
        source_address_prefix = "*"
        destination_address_prefix = "*"
    }
 
    tags = {
        "Name" = "InternetSG"
    }
}

Security Group Association

To tie the security groups to the subnets, use the resource, ‘azurerm_subnet_network_security_group_association’.  The definition consists of the subnet id defined above and the appropriate network security group id.

resource "azurerm_subnet_network_security_group_association" "public" {
    subnet_id =azurerm_subnet.PublicSubnet1.id
    network_security_group_id =azurerm_network_security_group.InternetSG.id
}

resource "azurerm_subnet_network_security_group_association" "private" {
    subnet_id =azurerm_subnet.PrivateSubnet1.id
    network_security_group_id =azurerm_network_security_group.OpenSG.id
}

Public IP

The Public IP is an Azure resource that allows a public IP to be held and not changed regardless of whether an instance goes offline or not.  Ephemeral public IP addresses change when an instance reboots, but the Public IP does not.  The Terraform resource below creates a public IP address that is static meaning that it will not allow any traffic unless allowed by a defined security group.

resource "azurerm_public_ip" "PublicIP" {
    name = "PublicIP"
    location = azurerm_resource_group.TestRG.location
    resource_group_name = azurerm_resource_group.TestRG.name
    allocation_method = "Static"
}

Network Interface Definition

For the CloudEOS instance and the host that will be created, both the public and private interfaces for the router and the private interface for the host need to be defined.

For the public interface in this example, it needs to be correlated to the PublicSubnet1 defined above.  A private IP is optional for the interface but helpful when troubleshooting as this is the IP that will be assigned to the interface.  Since traffic will flow through this interface, ‘enable_ip_forwarding’ will need to be enabled.  Also, since the Public IP defined above will be used on this interface, the ‘public_ip_address_id’ is tied into the interface in the definition.

For the private interface on the CloudEOS router, it needs to be correlated to the PrivateSubnet1 defined above.  A static private IP is optional for the interface but helpful when troubleshooting as this is the IP that will be assigned to the interface.  Since traffic will flow through this interface, ‘enable_ip_forwarding’ will need to be enabled.  Notice that there is no public IP assigned to this interface.

The ‘HostPrivateInt’ will have similar information to the private interface on the CloudEOS router, but since it will not be forwarding traffic through the device and simply originating and terminating traffic, it has ‘enable_ip_forwarding’ disabled.

resource "azurerm_network_interface" "PublicInt" {
    name = "PublicInt"
    location                      = azurerm_resource_group.TestRG.location
    resource_group_name           = azurerm_resource_group.TestRG.name
    enable_accelerated_networking = true
    enable_ip_forwarding          = true

    ip_configuration {
        name = "PublicIPInt"
        subnet_id = azurerm_subnet.PublicSubnet1.id
        private_ip_address_allocation = "Static"
        private_ip_address = "10.2.0.10"
        public_ip_address_id = azurerm_public_ip.PublicIP.id
    }
}

resource "azurerm_network_interface" "PrivateInt" {
    name = "PrivateInt"
    location                      = azurerm_resource_group.TestRG.location
    resource_group_name           = azurerm_resource_group.TestRG.name
    enable_accelerated_networking = true
    enable_ip_forwarding          = true

    ip_configuration {
        name = "PrivateIPInt"
        subnet_id = azurerm_subnet.PrivateSubnet1.id
        private_ip_address_allocation = "Static"
        private_ip_address = "10.2.1.10"
    }
}

resource "azurerm_network_interface" "HostPrivateInt" {
    name = "HostPrivateInt"
    location                      = azurerm_resource_group.TestRG.location
    resource_group_name           = azurerm_resource_group.TestRG.name
    enable_accelerated_networking = true
    enable_ip_forwarding          = false

    ip_configuration {
        name = "HostPrivateIPInt"
        subnet_id = azurerm_subnet.PrivateSubnet1.id
        private_ip_address_allocation = "Static"
        private_ip_address = "10.2.1.11"
    }
}

Route Table and Routes

Route tables and routes will need to be defined for both the public and private subnets, routes will need to be created, and the route tables will need to be associated with the appropriate subnets.

Define Route Tables and Routes

The two resource definitions below define the public and private route tables to be created.

Routes will need to be defined in the route tables specifying where the default traffic will be sent.  For the public route table, the default route (0.0.0.0/0) will be sent to the Internet. 

For the private route table, the default route will be sent to the CloudEOS router which is a virtual appliance, therefore, the ‘next_hop_type’ will be virtual appliance, and an IP address is needed as defined in the ‘next_hop_in_ip_address’ attribute.

resource "azurerm_route_table" "PublicRouteTable" {
    name = "PublicRT"
    location = azurerm_resource_group.TestRG.location
    resource_group_name = azurerm_resource_group.TestRG.name
    disable_bgp_route_propagation = false

    route {
        address_prefix = "0.0.0.0/0"
        name = "default_route"
        next_hop_type = "Internet"
    }
}

resource "azurerm_route_table" "PrivateRouteTable" {
    name = "PrivateRT"
    location = azurerm_resource_group.TestRG.location
    resource_group_name = azurerm_resource_group.TestRG.name
    disable_bgp_route_propagation = false

    route {
        address_prefix = "0.0.0.0/0"
        name = "default_route"
        next_hop_type = "VirtualAppliance"
        next_hop_in_ip_address = "10.2.0.10"
    }
}

Associate Subnets to the Route Tables

Once the route tables and routes are created, Azure needs to associate the route tables to the correct subnets.  For the public route table, it will be associated to the PublicSubnet1 subnet defined above.  For the private route table, it will be associated to the PrivateSubnet1 subnet.

resource "azurerm_subnet_route_table_association" "publicRtSubnetMap" {
    subnet_id = azurerm_subnet.PublicSubnet1.id 
    route_table_id = azurerm_route_table.PublicRouteTable.id
}

resource "azurerm_subnet_route_table_association" "privateRtSubnetMap" {
    subnet_id = azurerm_subnet.PrivateSubnet1.id 
    route_table_id = azurerm_route_table.PrivateRouteTable.id
}

CloudEOS Definition

This section will define the CloudEOS instance parameters and the bootstrap image.

CloudEOS Instance

The CloudEOS instance is ready to be defined.  The resource ‘azurerm_virtual_machine’ defines the CloudEOS1 instance and has several parameters that need to be present.

The ‘vm_size’ is needed to size the deployment of this instance.

Both network interface ids should be defined in the ‘network_interface_ids’ section.  The primary interface that will talk to the Internet should be defined as the ‘primary_network_interface_id’.

For the ‘storage_reference’ section of the CloudEOS creation, the ‘publisher’ is “arista-networks”, the ‘offer’ is “cloudeos-router-payg”, and the ‘sku’ is “cloudeos-4_24_0-payg-free” with ‘version’ “4.24.01”.

Use the parameters listed for the storage_os_disk, and then define the os_profile.  For the os_profile, note the parameters required for the ‘username’ and ‘password’ in Azure.  Also, in the ‘custom_data’ parameter, include the pointer to the bootstrap file. defined later in this section.  NOTE: The bootstrap.cfg file contains the familiar router configuration seen in the section below.

For this instance definition, the ‘os_profile_linux_config’ will not disable password authentication.  Lastly, at the bottom of this section is a storage account definition that is needed for the CloudEOS instance.  Make sure it is included in the Terraform file, and it will be referenced in the ‘boot_diagnostics’ section of the CloudEOS instance creation.

resource "azurerm_virtual_machine" "CloudEOS1" {
    name = "CloudEOS-VM"
    location = azurerm_resource_group.TestRG.location
    resource_group_name = azurerm_resource_group.TestRG.name
    network_interface_ids = [azurerm_network_interface.PublicInt.id, azurerm_network_interface.PrivateInt.id]
    primary_network_interface_id = azurerm_network_interface.PublicInt.id
    vm_size = "Standard_D2_v2"
    delete_os_disk_on_termination = true

    storage_image_reference {
        publisher = "arista-networks"
        offer = "cloudeos-router-payg"
        sku = "cloudeos-4_24_0-payg-free"
        version = "4.24.01"
    }

    storage_os_disk {
        name = "CloudEOSDisk1"
        caching = "ReadWrite"
        create_option = "FromImage"
        managed_disk_type = "Standard_LRS"
    }

    os_profile {
        computer_name = "CloudEOS-rtr"
        admin_username = "testuser"
        admin_password = "test!123456"
        custom_data = file("bootstrap.cfg")
    }

    plan {
        name = "cloudeos-4_24_0-payg-free"
        publisher = "arista-networks"
        product = "cloudeos-router-payg"
    }

    os_profile_linux_config {
        disable_password_authentication = false 
    }

    boot_diagnostics {
        enabled = true
        storage_uri = azurerm_storage_account.RouterStorage.primary_blob_endpoint
    }
}

resource "azurerm_storage_account" "RouterStorage" {
    name = "testrouterstorage"
    location = azurerm_resource_group.TestRG.location
    resource_group_name = azurerm_resource_group.TestRG.name
    account_tier = "Standard"
    account_replication_type = "LRS"
}

Sample Bootstrap Configuration for the CloudEOS Instance

Below is a sample bootstrap that could be loaded into the CloudEOS instance on boot.  For this example, this configuration would be located in a bootstrap.cfg text file as referenced above.

%EOS-STARTUP-CONFIG-START%
hostname CloudEOSTest
username testuser privilege 15 secret arista123
aaa authorization exec default local
ip routing
ip route 0.0.0.0/0 10.1.0.1
!
ip nat profile snat-for-leaf-routers
   ip nat source dynamic access-list acl-snat-for-leaves overload
!
ip access-list acl-snat-for-leaves
   10 permit ip any any
!
interface ethernet 1
   ip address 10.1.0.10/24
   ip nat service-profile snat-for-leaf-routers
!
interface ethernet 2
   ip address 10.1.1.10/24
!
%EOS-STARTUP-CONFIG-END%

Host

The host in this example is another CloudEOS instance that just has one interface defined on it.  The bootstraphost.cfg is similar to the one above and is provided as an example below.

resource "azurerm_virtual_machine" "CloudEOSHost" {
    name = "CloudEOSHost"
    location = azurerm_resource_group.TestRG.location
    resource_group_name = azurerm_resource_group.TestRG.name
    network_interface_ids = [azurerm_network_interface.HostPrivateInt.id]
    primary_network_interface_id = azurerm_network_interface.HostPrivateInt.id
    vm_size = "Standard_D2_v2"
    delete_os_disk_on_termination = true

    storage_image_reference {
        publisher = "arista-networks"
        offer = "cloudeos-router-payg"
        sku = "cloudeos-4_24_0-payg-free"
        version = "4.24.01"
    }

    storage_os_disk {
        name = "CloudEOSHostDisk1"
        caching = "ReadWrite"
        create_option = "FromImage"
        managed_disk_type = "Standard_LRS"
    }

    os_profile {
        computer_name = "CloudEOSHost"
        admin_username = "testuser"
        admin_password = "test!123456"
        custom_data = file("bootstraphost.cfg")
    }
 
    plan {
        name = "cloudeos-4_24_0-payg-free" 
        publisher = "arista-networks"
        product = "cloudeos-router-payg"
    }

    os_profile_linux_config {
        disable_password_authentication = false
    }

    boot_diagnostics {
        enabled = true
        storage_uri = azurerm_storage_account.HostStorage.primary_blob_endpoint
    }
}

resource "azurerm_storage_account" "HostStorage" {
    name = "hostcloudeosstorage"
    location = azurerm_resource_group.TestRG.location
    resource_group_name = azurerm_resource_group.TestRG.name 
    account_tier = "Standard"
    account_replication_type = "LRS"
}

Sample Bootstrap Configuration for the Host

Below is a sample bootstrap that is used for the host in this example.  A standard compute resource could be used instead as desired.  For this example, this configuration would be located in a bootstraphost.cfg text file as referenced above.  Note that ‘interface ethernet 1’ is missing from this configuration as it will get its address from DHCP, so you would need to get the address from the Azure console or use output from Terraform which is included in the outputs below.

%EOS-STARTUP-CONFIG-START%
hostname Host
username testuser privilege 15 secret arista123
ip routing
ip route 0.0.0.0/0 10.1.1.10
!
%EOS-STARTUP-CONFIG-END%

Output

In Terraform, an output can be defined to provide the user with information about a created resource.  In this case, the output provides the public IP address of the CloudEOS instance to allow the user to SSH into the device after creation.

output "PublicIP" {
    value =azurerm_public_ip.PublicIP.ip_address
}

output "HostIP" {
    value =azurerm_network_interface.HostPrivateInt.private_ip_address
}

Running the Terraform Script

Now that the script is complete, follow the following steps to create the instances in Azure

1. Login to the Azure CLI from a terminal window.  In many cases, it will be ‘az login’
2. From the directory where the main.tf, bootstrap.cfg, and bootstraphost.cfg files are, run ‘terraform init’
3. Next, run ‘terraform plan’ to make sure there are no errors in the file.
4. Run ‘terraform apply’ to begin the process of deploying your topology.  You will be asked to type ‘yes’ for your confirmation to build the resources in Azure.  Once this is complete, Terraform will provide the IP address from the output for you to connect to the CloudEOS instance in Azure.  Use the username/password defined in the bootstrap files for the credentials to login to the instance.
5. Run ‘terraform destroy’ to tear down the topology in Azure.  You will be asked to type ‘yes’ for your confirmation to remove the resources in Azure.

Additional Arista Terraform Example Material

When you are ready to learn about how to create multi-region solutions, enable segmentation from the data center to the cloud, incorporate CloudHA, and tie everything back into CloudVision as a Service, all using Terraform and EOS, visit Arista’s Github page at https://github.com/aristanetworks/CloudEOS