Load Balancing with ECMP: Hardware Configuration Lookup

Abstract:

This publication illustrates a technique which can be used to find exactly how Arista devices program routes to send traffic across multiple available paths. An example will be given on the Arista DCS-7150S-52-CL-R running EOS version 4.14.8M.

Initial configuration:

As an IGP we are using OSPF with maximum paths feature configured:

Arista(config)#router ospf 1
Arista(config-router-ospf)#maximum-paths 32

There are two iBGP peers configured via a peer-group “pg1”:

Arista(config)#router bgp 65001
Arista(config-router-bgp)#neighbor pg1 maximum-routes 16000 
Arista(config-router-bgp)#neighbor 172.20.18.49 peer-group pg1 
Arista(config-router-bgp)#neighbor 172.20.18.121 peer-group pg1

iBGP advertisements:

* >   10.82.2.32/27       172.20.16.143    0       100     0       64920 64944 i Or-ID: 172.20.16.143 C-LST: 172.20.16.129
*     10.82.2.32/27       172.20.16.143    0       100     0       64920 64944 i Or-ID: 172.20.16.143 C-LST: 172.20.16.130

Routing table contains two equal cost paths to the destination with reference to BGP:

B I    10.82.2.32/27 [200/0]   via 172.20.18.49,  Vlan102
                               via 172.20.18.121, Vlan103

Question:

Does it mean that the traffic is being load shared across two paths? If so, is there a way to show how it is programmed in hardware? Which interfaces are used? Which load balancing algorithm is used?

Recursive lookup for the actual path:

In Software:

Arista#show tech-support ribd | no-more
Nexthop:172.20.16.143 Flags: 0 Interface: 0(NULL)
<output ommited>
metric:30metric2:0
IGP Nexthops:
172.20.18.49
172.20.18.121
Routes using this Nexthop: <- two entries for the same destination
dest: 10.82.2.32/27  Instance: 3   
dest: 10.82.2.32/27  Instance: 3
Arista#show platform fm6000 l3 hardware routes
Prefix: 10.82.2.32/27, NextHop: 230   <- only one, so does it use ECMP?

The nexthop for destination is reachable via:

Arista#show ip route host
F  172.20.16.143 via 172.20.18.49 on Vlan102
                 via 172.20.18.121 on Vlan103

In Hardware:

Arista#show platform fm6000 l3 hardware routes
Prefix: 172.20.16.143/32, NextHop: 230

Original destination:

Arista#show platform fm6000 l3 hardware routes
Prefix: 10.82.2.32/27, NextHop: 230

We can see that the route has been originated by the router 172.20.16.143 and the iBGP advertisement has been replicated and arrived as two separate instances with cluster IDs 172.20.16.130 and 172.20.16.129. The BGP nexthop specified in both instances: 172.20.16.143

Recursively resolving the next hop to 172.20.16.143. The routing table is populated with two entries from OSPF with maximum-path configured option.

The nexthops are: 172.20.18.49 and 172.20.18.121 for directly connected routes on VLANs 102 and 103 respectively on interfaces Eth46 and Eth50.

What is it this NextHop: 230?

Arista#show platform fm6000 l3 hardware next-hop
NextHop: 230, Size: 2 Wide: 0
HwIndex: 398, Type: Arp, Mac: 001C7327A524, Vlan: 103, HwData: 0746001C7327A524, HitBit: 0
HwIndex: 399, Type: Arp, Mac: 001C732763EA, Vlan: 102, HwData: 0738001C732763EA, HitBit: 0

Resolving MACs:

Arista # show arp
172.20.18.49            0  001c.7327.63ea  Vlan102, Ethernet50
172.20.18.121           0  001c.7327.a524  Vlan103, Ethernet46

Conclusion:

The default BGP behaviour without the configuration of maximum paths option is such that only one entry will be allowed. Therefore, load sharing will not be used. Since the recursive resolution of the advertised BGP nexthop is also advertised by OSPF with the feature maximum-path configured, the load-sharing will be used to reach the advertised BGP nexthop. Nexthops for advertised BGP nexthop is on directly connected interface.

10.82.2.32/27 -> via iBGP
   172.20.16.143 -> via OSPF ECMP
      (172.20.18.121, 172.20.18.49) -> via Connected
         (001c.7327.a524  Vlan103, Ethernet46)   (001c.7327.a524  Vlan103, Ethernet46)

Which algorithm is used for load-balancing?

Arista#show port-channel load-balance fm6000 – if no profiles configured
Arista#show load-balance profile – if specific profile applied to the interface

Note: above applicable only on Alta-based platform: 7150S

Useful commands:

Arista#show tech-support ribd | no-more
Arista#show platform fm6000 l3 hardware routes
Arista#show platform fm6000 l3 hardware next-hop
Arista#show platform fm6000 tech-support | no-more