Implementing ECMP with BGP

Hi Raul,

The ECMP option in the maximum-paths command controls the maximum number of next-hops that get installed in the FIB after recursively looking up a next-hop for a prefix. By default this is already set to the platform maximum which you can check by running the command “show bgp instance | grep -i max”. In order to explain this command please see the following diagram

Topology

Here’s a summary of the depiction above

a) Sw1-Lo0_1.1.1.1(AS65000) <–eBGP—-> Sw2-Lo0_2.2.2.2(AS65001)
b) Sw1-Lo0_1.1.1.1(AS65000) <–eBGP—-> Sw3-Lo0_3.3.3.3(AS65001)
c) Sw2-Lo0_2.2.2.2(AS65001) <–iBGP—-> Sw3-Lo0_3.3.3.3(AS65001)
d) Sw3 is configured to advertise its Lo1 ip 33.33.33.33/32 into BGP
e) All BGP Peerings are configured between Loopback addresses
f) There are static routes for the respective loopbacks on all the switches

Sw1 Baseline Configs

sw1#show run section bgp
router bgp 65000
   distance bgp 20 200 200
   neighbor 2.2.2.2 remote-as 65001
   neighbor 2.2.2.2 update-source Loopback0
   neighbor 2.2.2.2 ebgp-multihop
   neighbor 2.2.2.2 maximum-routes 12000 
   neighbor 3.3.3.3 remote-as 65001
   neighbor 3.3.3.3 update-source Loopback0
   neighbor 3.3.3.3 ebgp-multihop
   neighbor 3.3.3.3 maximum-routes 12000 
sw1#

Sw1 BGP & Route Table

sw1#sh bgp instance | grep -i max
Maximum paths: 1 Maximum ECMP: 32
sw1#
sw1#sh ip bgp
BGP routing table information for VRF default
Router identifier 192.85.2.254, local AS number 65000
Route status codes: s - suppressed, * - valid, > - active, # - not installed, E - ECMP head, e - ECMP
                    S - Stale, c - Contributing to ECMP, b - backup, L - labeled-unicast
Origin codes: i - IGP, e - EGP, ? - incomplete
AS Path Attributes: Or-ID - Originator ID, C-LST - Cluster List, LL Nexthop - Link Local Nexthop

        Network                Next Hop              Metric  LocPref Weight  Path
 * >    33.33.33.33/32         3.3.3.3               0       100     0       65001 i
 *      33.33.33.33/32         2.2.2.2               0       100     0       65001 i
sw1#
sw1#sh ip route | egrep -A100 ^" C"
 C      1.1.1.1/32 is directly connected, Loopback0
 S      2.2.2.2/32 [1/0] via 10.1.0.1, Ethernet1
                         via 10.1.0.3, Ethernet2
 S      3.3.3.3/32 [1/0] via 10.1.0.5, Ethernet3
 C      10.1.0.0/31 is directly connected, Ethernet1
 C      10.1.0.2/31 is directly connected, Ethernet2
 C      10.1.0.4/31 is directly connected, Ethernet3
 B E    33.33.33.33/32 [20/0] via 10.1.0.5, Ethernet3
 C      192.85.2.0/24 is directly connected, Ethernet21

sw1#

Based on the above, when the default maximum-paths is used on this platform, we are seeing only a single route entry in the Routing Table.

 B E    33.33.33.33/32 [20/0] via 10.1.0.5, Ethernet3

Now, we shall modify the maximum-paths in order to see both paths, via Sw2 and Sw3 and also add the ecmp option.

sw1#conf t
sw1(config)#router bgp 65000
sw1(config-router-bgp)#maximum-paths 2 ecmp 2
sw1(config-router-bgp)#end
sw1#sh run section bgp
router bgp 65000
   distance bgp 20 200 200
   maximum-paths 2 ecmp 2
   neighbor 2.2.2.2 remote-as 65001
   neighbor 2.2.2.2 update-source Loopback0
   neighbor 2.2.2.2 ebgp-multihop
   neighbor 2.2.2.2 maximum-routes 12000 
   neighbor 3.3.3.3 remote-as 65001
   neighbor 3.3.3.3 update-source Loopback0
   neighbor 3.3.3.3 ebgp-multihop
   neighbor 3.3.3.3 maximum-routes 12000 
sw1#

Now, if we look at the BGP Table we shall see both paths chosen and contributing towards ECMP and we shall see both of them in the Routing Table as different next-hops (a.k.a VIAs) for the route 33.33.33.33/32.

sw1#sh ip bgp
BGP routing table information for VRF default
Router identifier 192.85.2.254, local AS number 65000
Route status codes: s - suppressed, * - valid, > - active, # - not installed, E - ECMP head, e - ECMP
                    S - Stale, c - Contributing to ECMP, b - backup, L - labeled-unicast
Origin codes: i - IGP, e - EGP, ? - incomplete
AS Path Attributes: Or-ID - Originator ID, C-LST - Cluster List, LL Nexthop - Link Local Nexthop

        Network                Next Hop              Metric  LocPref Weight  Path
 * >Ec  33.33.33.33/32         2.2.2.2               0       100     0       65001 i
 *  ec  33.33.33.33/32         3.3.3.3               0       100     0       65001 i
sw1#
sw1#sh ip route | egrep -A100 ^" C"
 C      1.1.1.1/32 is directly connected, Loopback0
 S      2.2.2.2/32 [1/0] via 10.1.0.1, Ethernet1
                         via 10.1.0.3, Ethernet2
 S      3.3.3.3/32 [1/0] via 10.1.0.5, Ethernet3
 C      10.1.0.0/31 is directly connected, Ethernet1
 C      10.1.0.2/31 is directly connected, Ethernet2
 C      10.1.0.4/31 is directly connected, Ethernet3
 B E    33.33.33.33/32 [20/0] via 10.1.0.1, Ethernet1
                              via 10.1.0.5, Ethernet3
 C      192.85.2.0/24 is directly connected, Ethernet21

sw1#

However, the route to 2.2.2.2 which is the BGP next-hop for the prefix 33.33.33.33/32 has two next-hop entries in the Routing table based on the static route, but we are only seeing the one via 10.1.0.1 being installed.

 S      2.2.2.2/32 [1/0] via 10.1.0.1, Ethernet1
                         via 10.1.0.3, Ethernet2

Because we have the ecmp parameter set to 2, the other next-hop via 10.1.0.3 is not installed. We only see 33.33.33.33 reachable via 10.1.0.1 and 10.1.0.5

 B E    33.33.33.33/32 [20/0] via 10.1.0.1, Ethernet1
                              via 10.1.0.5, Ethernet3

This is what the ecmp parameter controls.

If we modify the maximum-paths command such that we do not specify the ecmp parameter and leave it to be the platform default, which in this case is 32 then we shall see that for the BGP route we have a total of 3 next-hops VIAs (next-hop entries) in the Routing Table.

sw1(config)#router bgp 65000
sw1(config-router-bgp)#no maximum-paths 2 ecmp 2
sw1(config-router-bgp)#maximum-paths 2
sw1(config-router-bgp)#sh active
router bgp 65000
   distance bgp 20 200 200
   maximum-paths 2
   neighbor 2.2.2.2 remote-as 65001
   neighbor 2.2.2.2 update-source Loopback0
   neighbor 2.2.2.2 ebgp-multihop
   neighbor 2.2.2.2 maximum-routes 12000 
   neighbor 3.3.3.3 remote-as 65001
   neighbor 3.3.3.3 update-source Loopback0
   neighbor 3.3.3.3 ebgp-multihop
   neighbor 3.3.3.3 maximum-routes 12000 
sw1(config-router-bgp)#
sw1(config-router-bgp)#end
sw1#sh bgp instance | grep -i max
Maximum paths: 2 Maximum ECMP: 32
sw1#

We still see both paths as ECMP contributors in the BGP Table due to maximum-paths 2

sw1#sh ip bgp
BGP routing table information for VRF default
Router identifier 192.85.2.254, local AS number 65000
Route status codes: s - suppressed, * - valid, > - active, # - not installed, E - ECMP head, e - ECMP
                    S - Stale, c - Contributing to ECMP, b - backup, L - labeled-unicast
Origin codes: i - IGP, e - EGP, ? - incomplete
AS Path Attributes: Or-ID - Originator ID, C-LST - Cluster List, LL Nexthop - Link Local Nexthop

        Network                Next Hop              Metric  LocPref Weight  Path
 * >Ec  33.33.33.33/32         2.2.2.2               0       100     0       65001 i
 *  ec  33.33.33.33/32         3.3.3.3               0       100     0       65001 i
sw1#

However, now in the Routing Table we see all 3 next-hops because the ecmp parameter is set to the platform default of 32.

sw1#sh ip route | egrep -A100 ^" C"
 C      1.1.1.1/32 is directly connected, Loopback0
 S      2.2.2.2/32 [1/0] via 10.1.0.1, Ethernet1
                         via 10.1.0.3, Ethernet2
 S      3.3.3.3/32 [1/0] via 10.1.0.5, Ethernet3
 C      10.1.0.0/31 is directly connected, Ethernet1
 C      10.1.0.2/31 is directly connected, Ethernet2
 C      10.1.0.4/31 is directly connected, Ethernet3
 B E    33.33.33.33/32 [20/0] via 10.1.0.1, Ethernet1
                              via 10.1.0.3, Ethernet2
                              via 10.1.0.5, Ethernet3
 C      192.85.2.0/24 is directly connected, Ethernet21

sw1#

HTH