• Hybrid cloud connectivity with Arista’s Extensible Operating System (EOS) and Amazon Web Services (AWS)

 
 
Print Friendly, PDF & Email

Motivation

The term Hybrid Cloud is not clearly defined but the most common definition is a scenario where a customer wants to combine resources in their own data centre (the private cloud) with resources in the public cloud. To allow these services to communicate, a connection between the two cloud environments needs to be established. Most cloud providers offer two options: An Internet Protocol Security (IPsec)-based Virtual Private Network (VPN) connection using the Internet as a transport media and a private link using dedicated lines or equivalent technology. In this article, we cover the VPN based approach using EOS-based services as a customer VPN gateway and AWS as a public cloud provider.

Includes and excludes

This tutorial gives an overview of how to configure AWS and EOS instances to establish IPSec encrypted tunnels between AWS and an on-premises data centre including dynamic routing. The tutorial typically uses default parameters for IPSec encryption or the dynamic routing protocol. Please refer to your favourite IPSec and Border Gateway Protocol (BGP) tutorial as well as the AWS documentation centre and the EOS manual for further details on the subject.

Solution components overview

Our demo features a typical AWS setup consisting of a Virtual Private Cloud (VPC) in one AWS region with two subnets assigned to two different Availability Zones (AZ) and associated to one routing table. Assigned to each subnet is one instance of AWS’s Elastic Compute Cloud (EC2) service. They will act as ping targets for services running in our private cloud.

Public cloud details

Please note that we omit some details if they are not directly relevant to our demo setup (like for example security groups).

Public cloud details

VPC details

VPC AWS region Classless Inter-Domain Routing (CIDR) block
vpc-hybridcloud eu-central-1 10.1.0.0/16

Subnet details

Subnet AZ CIDR Routing table
vpc-hybridcloud-subnet-10-1-1-0 eu-central-1a 10.1.1.0/24 vpc-hybridcloud-rt
vpc-hybridcloud-subnet-10-1-2-0 eu-central-1b 10.1.2.0/24 vpc-hybridcloud-rt

EC2 instance details

Instance Private IP address Subnet
host1 10.1.1.10 vpc-hybridcloud-subnet-10-1-1-0
host2 10.1.2.10 vpc-hybridcloud-subnet-10-1-2-0

Private cloud details

Private cloud details

The private cloud consists of three routers. One router represents our data centre. In reality, it could, for example, be a Layer 3 Leaf and Spine (L3LS) fabric with an EVPN-VXLAN (Ethernet VPN with Virtual Extensible LAN data plane) overlay, consisting of a large number of switches. Attached to the data centre router there are two subnets, 10.2.1.0/24 and 10.2.2.0/24, each containing one host acting as ping sources. Besides, there are two customer gateway routers. Each router is connected to the data centre router and has got an Internet uplink with a public IP address. These two routers will terminate the IPSec tunnels from AWS to our private cloud. Note: In reality, you would want to secure the Internet-facing interface with a firewall or Access Control Lists (ACL). You can also move all interfaces representing your internal network to custom Virtual routing and forwarding (VRF) instances. These implementation and customer-specific details are beyond the scope of this tutorial, though.

To date, either the 7020SRG hardware platform or the vEOS Router software platform can be used as an on-premises IPSec gateway.

Router instances including autonomous system number (ASN)

Router name Role ASN
dc Data centre router (simulating data centre fabric) 65000
cgw1 Customer gateway 65001
cgw2 Customer gateway 65002

Topology, subnets and interfaces

IP network Function Router 1 Router 1 IP address Router 1 interface Router 2 Router 2 IP address Router 2 interface
10.2.1.0/24 Data centre host network dc 10.2.1.1 Ethernet3
10.2.2.0/24 Data centre host network dc 10.2.2.1 Ethernet4
10.2.3.0/24 Point to point network cgw1 10.2.3.1 Ethernet2 dc 10.2.3.2 Ethernet1
10.2.4.0/24 Point to point network cgw2 10.2.4.1 Ethernet2 dc 10.2.4.2 Ethernet2
XXX.XXX.231.192/26 Point to point network cgw1 XXX.XXX.231.224 Ethernet1 Internet router XXX.XXX.231.193
XXX.XXX.10.176/29 Point to point network cgw2 XXX.XXX.10.179 Ethernet1 Internet router XXX.XXX.10.177

Hosts attached to the data centre router

Host IP address
host3 10.2.1.10
host4 10.2.2.10

Interconnecting the public and the private cloud

AWS offers two options to establish IPSec tunnels to remote locations:

  1. A managed VPN service called VPN gateway

  2. Running your own VPN gateway as an EC2 instance

There are pros and cons of each solution which are beyond the scope of this tutorial. In this article, we use the AWS provided VPN gateway option but will cover the usage of vEOS Router instances in a follow-up article.

The VPN gateway option allows us to choose between static routing and using BGP as a dynamic routing protocol. Due to the increased flexibility, we are going to use BGP but the solution can easily be adapted (simplified, in fact) to use static routing instead.

Solution overview

A VPN tunnel is established between two tunnel endpoints. In our scenario, both tunnel endpoints use fixed IP addresses. To provide redundancy, different approaches are used:

  • The tunnel endpoint address is shared by more than one device. This approach is typically used by firewalls joined to a high-available firewall cluster, which typically shares the security association (SA) states between the cluster members

  • Two tunnels are established and the tunnel state monitored by, for example, the Dead Peer Detection (DPD) feature. Alternatively or in addition to DPD, a dynamic routing protocol, which usually exchanges hello packets on a regular basis between both sides, can be used to determine if a tunnel is still alive or not

AWS’ virtual gateway takes the latter approach. AWS provides the user with two tunnel endpoint addresses and at least two tunnels must be configured to provide redundancy. EOS devices use the same approach and two tunnel endpoints on the private cloud side are configured. We configure a full-mesh between the two tunnel endpoints on the AWS side and the two tunnel endpoints on our side.

As already mentioned, the BGP protocol is used the exchange routes between AWS and our local premises.

VPN topology

AWS configuration

We now perform all required configuration steps within the AWS environment. All subsequent steps happen in the context of the VPC service console.

Defining the customer gateways

For each of the two customer gateway routers, we define a customer gateway object. We also define the ASN for each system.

  1. In the AWS console, select the VPC feature and select Virtual Private Network (VPN) > Customer Gateways and click on Create Customer Gateway
  2. Provide the details for the customer gateway 1 and click on Create Customer Gateway
  3. Close the confirmation screen and click on Create Customer Gateway again to create the 2nd gateway:
  4. Provide the details for the customer gateway 2 and click on Create Customer Gateway
  5. Close the confirmation screen. Both gateways should now be visible and in the state available

Defining the VPN gateway

We create the VPN gateway object using default parameters and associate it with our VPC.

  1. In the AWS console, select the VPC feature and select Virtual Private Network (VPN) > Virtual Private Gateways and click on Create Virtual Private Gateway
  2. Provide the name for the VPN gateway and select Amazon default ASN (that equals to ASN 64512) and click on Create Virtual Private Gateway
  3. Close the confirmation screen. The VPN gateway should now be visible and in the state detached. Make sure the VPN gateway is selected and from the Actions menu choose Attach to VPC
  4. Select the VPC and confirm the selection by clicking on Yes, Attach
  5. After waiting a few minutes the VPN gateway should be in the state attached

Defining the Site to Site VPN connections

We create a VPN tunnel to each of our two customer gateways and download the configuration instructions, which include the IP addresses of the VPN gateway as well as the pre-shared keys. These values are later needed when we configure the customer gateway routers.

  1. In the AWS console, select the VPC feature and select Virtual Private Network (VPN) > Site-to-Site VPN Connections and click on Create VPN Connection
  2. Provide the necessary data (we use many of AWS’ default values) and click on Create VPN Connection
  3. Close the confirmation screen.  After waiting a few minutes the VPN connection should be available. Click on Create VPN connection to create the second VPN connection
  4. Provide the necessary data (we use many of AWS’ default values) and click on Create VPN Connection
  5. Close the confirmation screen.  After waiting a few minutes the VPN connection should be available
  6. Select each VPN connection on after each other and download the configuration using Generic as both the vendor as well as the platform. The relevant values are summarised in the table below:
Tunnel Name AWS VPN gateway Local gateway AWS endpoint Local endpoint Pre-shared key
Tunnel 1 vpn-to-cgw1 vgw cgw1 3.123.137.35 XXX.XXX.231.224 47aKHugpe4AzTRwa8bIYIr.eEY5UXjr3
Tunnel 2 vpn-to-cgw1 vgw cgw1 35.157.164.38 XXX.XXX.231.224 yGLTJwxfB3ahdGpGBlHw2AEfzUvPgA
Tunnel 3 vpn-to-cgw2 vgw cgw2 3.121.81.239 XXX.XXX.10.179 lnKZBK2F2PSoWEW62ABVx_KxbTaBxWX9
Tunnel 4 vpn-to-cgw2 vgw cgw2 18.195.198.253 XXX.XXX.10.179 yw2rbbdGfnhCDst1.bx0gEPeeU1U_muq
Tunnel Point to point network Remote IP address Local IP address Remote ASN Local ASN
Tunnel 1 169.254.15.208/30 169.254.15.209 169.254.15.210 64512 65001
Tunnel 2 169.254.216.120/30 169.254.216.121 169.254.216.122 64512 65001
Tunnel 3 169.254.29.92/30 169.254.29.93 169.254.29.94 64512 65002
Tunnel 4 169.254.122.56/30 169.254.122.57 169.254.122.58 64512 65002

Enable dynamic routing in the routing table

By default, routes learned via a dynamic routing protocol are not installed in the routing table. This needs to be enabled.

  1. In the AWS console, select the VPC feature and select Virtual Private Cloud > Route Tables, select the routing table and the Route Propagation tab and click on Edit route propagation
  2. Select the Propagate check box next to our previously defined VPN gateway and click on Save

EOS configuration

The configuration of the two customer gateway routers, as well as the data centre router, is as follows:

Configure data centre router dc

The data centre router is configured with the IP addresses of all four interfaces and BGP with the two customer gateway routers as neighbours.

Interfaces and BGP routing within the private cloud

interface Ethernet1
   no switchport
   ip address 10.2.3.2/24
!
interface Ethernet2
   no switchport
   ip address 10.2.4.2/24
!
interface Ethernet3
   no switchport
   ip address 10.2.1.1/24
!
interface Ethernet4
   no switchport
   ip address 10.2.2.1/24
!
ip routing
!
router bgp 65000
   neighbor 10.2.3.1 remote-as 65001
   neighbor 10.2.3.1 maximum-routes 12000 
   neighbor 10.2.4.1 remote-as 65002
   neighbor 10.2.4.1 maximum-routes 12000 
   network 10.2.1.0/24
   network 10.2.2.0/24

Configure customer gateway 1 router cgw1

Licensing

The 7020SRG hardware platform requires an IPSec licence to be installed before the IPSec features can be used. The vEOS Router software platform requires an IPSec licence and a generic vEOS licence before it can be used.

The licences must be purchased and are shipped as a file to the customer from Arista’s customer service department. The files should be uploaded to the switch, for example to the /mnt/flash directory. Then, the licence can be installed from CLI via:

license import flash:FILENAME

The correct licence installation can be verified using the command

cgw1#show license 
Customer name:         XXX
System Serial number:  XXX
System MAC address:    XXXX.XXXX.XXXX
Domain name:           Unknown
Platform:              vEOS-KVM

License feature:  IPSec 
        License parameter:  None
        Count:              1
        Start:              2019-10-28 01:00:00
        Expiration:         2029-10-25 02:00:00
        Active:             yes


License feature:  vEOS  -  Virtualized EOS
        Throughput:         Unknown
        Count:              1
        Start:              2019-10-28 01:00:00
        Expiration:         2029-10-25 02:00:00
        Active:             yes

In this example, vEOS Router is used as customer gateway routers.

Interfaces and BGP routing within the private cloud

interface Ethernet1
   no switchport
   ip address XXX.XXX.231.224/26
!
interface Ethernet2
   no switchport
   ip address 10.2.3.1/24
!
ip route 0.0.0.0/0 XXX.XXX.231.193
!
ip routing
!
router bgp 65001
   neighbor 10.2.3.2 remote-as 65000
   neighbor 10.2.3.2 maximum-routes 12000

IPSec configuration

The IPSec configuration consists of the following elements:

  • IKE policy to define the parameters for phase 1

  • SA policy to define a set of security parameters

  • IPSec profile linking the IKE policy, the SA policy and the pre-shared key (PSK)

  • The tunnel interface defining the local and the remote gateway’s IP address, a tunnel IP address and referring to the IPSec profile as well as some parameters like TCP MSS ceiling

The security settings match what is configured on the AWS side. In this tutorial, we used the default settings but stronger cyphers can be selected.

ip security
   ike policy ike-policy-amazon-vpn
      integrity sha1
      dh-group 2
      version 1
   !
   sa policy sa-policy-amazon-vpn
      esp encryption aes128
      esp integrity sha1
      pfs dh-group 2
   !
   profile profile-amazon-vpn1
      ike-policy ike-policy-amazon-vpn 
      sa-policy sa-policy-amazon-vpn 
      connection start
      shared-key 47aKHugpe4AzTRwa8bIYIr.eEY5UXjr3
      dpd 10 30 restart
   !
   profile profile-amazon-vpn2
      ike-policy ike-policy-amazon-vpn 
      sa-policy sa-policy-amazon-vpn 
      connection start
      shared-key yGLTJwx_fB_3ahdGpGBlHw2AEfzUvPgA
      dpd 10 30 restart

Now, we need to configure the two tunnel interfaces (one tunnel to each of the two AWS tunnel endpoints):

interface Tunnel0
   mtu 1436
   ip address 169.254.15.210/30
   tcp mss ceiling ipv4 1379 egress
   tunnel mode ipsec
   tunnel source XXX.XXX.231.224
   tunnel destination 3.123.137.35
   tunnel ipsec profile profile-amazon-vpn1
!
interface Tunnel1
   mtu 1436
   ip address 169.254.216.122/30
   tcp mss ceiling ipv4 1379 egress
   tunnel mode ipsec
   tunnel source XXX.XXX.231.224
   tunnel destination 35.157.164.38
   tunnel ipsec profile profile-amazon-vpn2

Using tunnel interfaces with IPSec is called route-based IPSec (as opposed to the more traditional policy-based IPSec style). The benefit of using route-based IPSec is that there is a tunnel interface that can be used for dynamic routing (like we do in our example) but also for Quality of Service (QoS), applying ACLs, etc. In the past, devices often supported policy-based IPSec only. In that case, a GRE tunnel with corresponding GRE tunnel interfaces was used to achieve similar results. EOS devices support the GRE tunnel model as well but the plain route-based tunnel interface (as indicated by tunnel mode ipsec) produces less overhead. The tunnel interface’s MTU is aligned to the setting on the AWS side. Adjusting the TCP MSS size helps avoid fragmentation as TCP connections crossing the tunnel use a smaller size that can be transmitted without packet fragmentation. Note: EOS 4.23.0 or higher is required to perform TCP MSS adjustments on the 7020SRG hardware platform on plain IPSec tunnels.

Dynamic routing towards AWS

In the last configuration step, the AWS routers are configured as BGP neighbours:

router bgp 65001
   neighbor 169.254.15.209 remote-as 64512
   neighbor 169.254.15.209 maximum-routes 12000
   neighbor 169.254.216.121 remote-as 64512
   neighbor 169.254.216.121 maximum-routes 12000

Note: AWS defines 30 seconds as BGP hold time whereas, EOS, uses a default of 90 seconds. If both sides propose different values, the routers agree on the lower value. That is why we do not have to align the settings on our side although they can be included in the configuration if preferred.

Configure customer gateway 2 router cgw2

The customer gateway 2 router is configured identically to customer gateway 1 with the notable exception of the IP addresses, ASN and the IPSec parameters.

Licensing

Please refer to the licensing section of customer gateway 1 router for how to install the licence on the machine.

Interfaces and BGP routing within the private cloud

interface Ethernet1
   no switchport
   ip address XXX.XXX.10.179/29
!
interface Ethernet2
   no switchport
   ip address 10.2.4.1/24
!
ip route 0.0.0.0/0 XXX.XXX.10.179
!
ip routing
!
router bgp 65002
   neighbor 10.2.4.2 remote-as 65000
   neighbor 10.2.4.2 maximum-routes 12000

IPSec configuration

ip security
   ike policy ike-policy-amazon-vpn
      integrity sha1
      dh-group 2
      version 1
   !
   sa policy sa-policy-amazon-vpn
      esp encryption aes128
      esp integrity sha1
      pfs dh-group 2
   !
   profile profile-amazon-vpn1
      ike-policy ike-policy-amazon-vpn 
      sa-policy sa-policy-amazon-vpn 
      connection start
      shared-key lnKZBK2F2PSoWEW62ABVx_KxbTaBxWX9
      dpd 10 30 restart
   !
   profile profile-amazon-vpn2
      ike-policy ike-policy-amazon-vpn 
      sa-policy sa-policy-amazon-vpn 
      connection start
      shared-key yw2rbbdGfnhCDst1.bx0gEPeeU1U_muq
      dpd 10 30 restart
interface Tunnel0
   mtu 1436
   ip address 169.254.29.94/30
   tcp mss ceiling ipv4 1379 egress
   tunnel mode ipsec
   tunnel source XXX.XXX.10.179
   tunnel destination 3.121.81.239
   tunnel ipsec profile profile-amazon-vpn1
!
interface Tunnel1
   mtu 1436
   ip address 169.254.122.58/30
   tcp mss ceiling ipv4 1379 egress
   tunnel mode ipsec
   tunnel source XXX.XXX.10.179
   tunnel destination 18.195.198.253
   tunnel ipsec profile profile-amazon-vpn2

Dynamic routing towards AWS

router bgp 65002
   neighbor 169.254.29.93 remote-as 64512
   neighbor 169.254.29.93 maximum-routes 12000
   neighbor 169.254.122.57 remote-as 64512
   neighbor 169.254.122.57 maximum-routes 12000

Solution verification

AWS VPN status

AWS tunnel status

The AWS console must report status Up for all VPN tunnels.

  1. In the AWS console, select the VPC feature and select Virtual Private Network (VPN) > Site-to-Site VPN Connections and select both VPN connections one after each other. Make sure the Tunnel Details tab is selected. The Status must be reported as UP. Take also a note of the number of received BGP routes from the private cloud.

AWS routing table

The routing table must list both our data centre prefixes 10.2.1.0/24 and 10.2.2.0/24.

  1. In the AWS console, select the VPC feature and select Virtual Private Cloud > Route Tables, select the routing table and the Routes tab and take note of the routes received from our private cloud via BGP

EOS VPN STATUS

Verify tunnel operations

Verifying the connectivity can be done at multiple levels. First, the presence of active SAs (we run the commands on cgw1, the same can be done at cgw2):

cgw1#show ip security connection 
Tunnel     Source           Dest             Status       Uptime     Input            Output           Rekey Time  
Tunnel0    XXX.XXX.231.224  3.123.137.35     Established  1 hour     85002 bytes      88918 bytes      12 minutes  
                                                                     1368 pkts        1443 pkts                    
Tunnel1    XXX.XXX.231.224  35.157.164.38    Established  1 hour     79821 bytes      88819 bytes      11 minutes  
                                                                     1268 pkts        1441 pkts

Secondly, the tunnel interfaces must be up:

cgw1#show ip interface Tunnel 0-1 brief 
                                                                                Address 
Interface       IP Address               Status       Protocol           MTU    Owner   
--------------- ------------------------ ------------ -------------- ---------- ------- 
Tunnel0         169.254.15.210/30        up           up                1436            
Tunnel1         169.254.216.122/30       up           up                1436

Lastly, there must be a BGP adjacency to the AWS routers in state Established:

cgw1#show ip bgp summary 
BGP summary information for VRF default
Router identifier 10.2.3.1, local AS number 65001
Neighbor Status Codes: m - Under maintenance
  Neighbor         V  AS           MsgRcvd   MsgSent  InQ OutQ  Up/Down State   PfxRcd PfxAcc
  10.2.3.2         4  65000           1623      1622    0    0    1d02h Estab   2      2
  169.254.15.209   4  64512           3337      3338    0    0 02:03:10 Estab   3      3
  169.254.216.121  4  64512           3340      3340    0    0 02:03:10 Estab   3      3

EOS routing table

AWS should announce the VPC’s CIDR block as one aggregated prefix. The route must be visible as BGP route learned via the tunnel interfaces:

cgw1#show ip route 10.1.0.0/16 

VRF: default
Codes: C - connected, S - static, K - kernel, 
       O - OSPF, IA - OSPF inter area, E1 - OSPF external type 1,
       E2 - OSPF external type 2, N1 - OSPF NSSA external type 1,
       N2 - OSPF NSSA external type2, B - BGP, B I - iBGP, B E - eBGP,
       R - RIP, I L1 - IS-IS level 1, I L2 - IS-IS level 2,
       O3 - OSPFv3, A B - BGP Aggregate, A O - OSPF Summary,
       NG - Nexthop Group Static Route, V - VXLAN Control Service,
       DH - DHCP client installed default route, M - Martian,
       DP - Dynamic Policy Route, L - VRF Leaked

 B E      10.1.0.0/16 [200/100] via 169.254.15.209, Tunnel0

A similar command can be used to check that the prefix is learned via both tunnels:

cgw1#show ip bgp 10.1.0.0/16
BGP routing table information for VRF default
Router identifier 10.2.3.1, local AS number 65001
BGP routing table entry for 10.1.0.0/16
 Paths: 2 available
  64512
    169.254.15.209 from 169.254.15.209 (169.254.15.209)
      Origin IGP, metric 100, localpref 100, IGP metric 1, weight 0, received 02:09:18 ago, valid, external, best
      Rx SAFI: Unicast
  64512
    169.254.216.121 from 169.254.216.121 (169.254.216.121)
      Origin IGP, metric 200, localpref 100, IGP metric 1, weight 0, received 02:09:18 ago, valid, external
      Rx SAFI: Unicast

Pinging EC2 instances

We ping host1 and host2 from host3

host3> ping 10.1.1.10
84 bytes from 10.1.1.10 icmp_seq=1 ttl=252 time=40.096 ms
84 bytes from 10.1.1.10 icmp_seq=2 ttl=252 time=29.028 ms
84 bytes from 10.1.1.10 icmp_seq=3 ttl=252 time=28.977 ms
84 bytes from 10.1.1.10 icmp_seq=4 ttl=252 time=40.008 ms
84 bytes from 10.1.1.10 icmp_seq=5 ttl=252 time=27.468 ms
host3> ping 10.1.2.10
84 bytes from 10.1.2.10 icmp_seq=1 ttl=252 time=32.633 ms
84 bytes from 10.1.2.10 icmp_seq=2 ttl=252 time=29.788 ms
84 bytes from 10.1.2.10 icmp_seq=3 ttl=252 time=39.511 ms
84 bytes from 10.1.2.10 icmp_seq=4 ttl=252 time=30.972 ms
84 bytes from 10.1.2.10 icmp_seq=5 ttl=252 time=28.482 ms

We ping host1 and host2 as well from host4

host4> ping 10.1.1.10
84 bytes from 10.1.1.10 icmp_seq=1 ttl=252 time=39.997 ms
84 bytes from 10.1.1.10 icmp_seq=2 ttl=252 time=28.583 ms
84 bytes from 10.1.1.10 icmp_seq=3 ttl=252 time=35.025 ms
84 bytes from 10.1.1.10 icmp_seq=4 ttl=252 time=35.891 ms
84 bytes from 10.1.1.10 icmp_seq=5 ttl=252 time=32.954 ms
host4> ping 10.1.2.10
84 bytes from 10.1.2.10 icmp_seq=1 ttl=252 time=40.009 ms
84 bytes from 10.1.2.10 icmp_seq=2 ttl=252 time=29.565 ms
84 bytes from 10.1.2.10 icmp_seq=3 ttl=252 time=34.906 ms
84 bytes from 10.1.2.10 icmp_seq=4 ttl=252 time=31.507 ms
84 bytes from 10.1.2.10 icmp_seq=5 ttl=252 time=33.026 ms

Conclusion

This concludes the tutorial to implement an IPSec tunnel between an AWS VPC and a private cloud instance using EOS and with dynamic routing support.

Follow

Get every new post on this blog delivered to your Inbox.

Join other followers: