Posted on March 23, 2020 4:00 am
 |  Asked by Ashwin C S
 |  62 views
0
0
Print Friendly, PDF & Email

Hello,

I was trying to simulate symmetric IRB with EVPN and ran into an issue wherein the servers are unable to communicate. We need your help here.
Problem Description
1. Ping between Servers H1 and H4 is failing. Server H1 & H4 communicate with the help of EVPN symmetric IRB and VXLAN datapath.
EOS versions
4.21.6F (Arista EOS containers)
Topology 
 
unnamed (2).png
Troubleshooting steps carried out
1. The control plane works. The type 2 routes are being exchanged between NVEs L1 and L4.

root@t12_vm8:/home/ece792# docker exec -it L1 Cli -c “show bgp evpn route-type mac-ip 9a9f.854f.9c11”
BGP routing table information for VRF default
Router identifier 21.21.21.21, local AS number 21
Route status codes: s – suppressed, * – valid, > – active, # – not installed, E – ECMP head, e – ECMP
S – Stale, c – Contributing to ECMP, b – backup
% – Pending BGP convergence
Origin codes: i – IGP, e – EGP, ? – incomplete
AS Path Attributes: Or-ID – Originator ID, C-LST – Cluster List, LL Nexthop – Link Local Nexthop

Network               Next Hop Metric  LocPref Weight Path
* >Ec   RD: 24:24 mac-ip 9a9f.854f.9c11
24.24.24.24 – 100     0 100 24 i
*  ec RD: 24:24 mac-ip 9a9f.854f.9c11
24.24.24.24 – 100     0 200 24 i
* >Ec   RD: 24:24 mac-ip 9a9f.854f.9c11 10.10.24.40
24.24.24.24 – 100     0 100 24 i
*  ec RD: 24:24 mac-ip 9a9f.854f.9c11 10.10.24.40
24.24.24.24 – 100     0 200 24 i
root@t12_vm8:/home/ece792#

root@t12_vm8:/home/ece792# docker exec -it L4 Cli -c “show bgp evpn route-type mac-ip 16d1.4d40.3872”

BGP routing table information for VRF default

Router identifier 24.24.24.24, local AS number 24

Route status codes: s – suppressed, * – valid, > – active, # – not installed, E – ECMP head, e – ECMP

                    S – Stale, c – Contributing to ECMP, b – backup

                    % – Pending BGP convergence

Origin codes: i – IGP, e – EGP, ? – incomplete

AS Path Attributes: Or-ID – Originator ID, C-LST – Cluster List, LL Nexthop – Link Local Nexthop


         Network                Next Hop Metric  LocPref Weight Path

 * >Ec   RD: 21:21 mac-ip 16d1.4d40.3872

                                21.21.21.21           – 100 0   200 21 i

 *  ec   RD: 21:21 mac-ip 16d1.4d40.3872

                                21.21.21.21           – 100 0   100 21 i

 * >Ec   RD: 21:21 mac-ip 16d1.4d40.3872 10.10.21.10

                                21.21.21.21           – 100 0   200 21 i

 *  ec   RD: 21:21 mac-ip 16d1.4d40.3872 10.10.21.10

                                21.21.21.21           – 100 0   100 21 i

root@t12_vm8:/home/ece792#

2. The routes are being installed on the NVEs L1 and L4 :

root@t12_vm8:/home/ece792# docker exec -it L1 Cli -c “show ip route vrf tenant-a”


VRF: tenant-a

Codes: C – connected, S – static, K – kernel,

       O – OSPF, IA – OSPF inter area, E1 – OSPF external type 1,

       E2 – OSPF external type 2, N1 – OSPF NSSA external type 1,

       N2 – OSPF NSSA external type2, B I – iBGP, B E – eBGP,

       R – RIP, I L1 – IS-IS level 1, I L2 – IS-IS level 2,

       O3 – OSPFv3, A B – BGP Aggregate, A O – OSPF Summary,

       NG – Nexthop Group Static Route, V – VXLAN Control Service,

       DH – DHCP client installed default route, M – Martian,

       DP – Dynamic Policy Route, L – VRF Leaked


Gateway of last resort is not set


 C        10.10.21.0/24 is directly connected, Vlan21

 B E      10.10.24.40/32 [200/0] via VTEP 24.24.24.24 VNI 1000 router-mac b6:b4:7a:6b:e3:87

root@t12_vm8:/home/ece792# docker exec -it L4 Cli -c “show ip route vrf tenant-a”


VRF: tenant-a

Codes: C – connected, S – static, K – kernel,

       O – OSPF, IA – OSPF inter area, E1 – OSPF external type 1,

       E2 – OSPF external type 2, N1 – OSPF NSSA external type 1,

       N2 – OSPF NSSA external type2, B I – iBGP, B E – eBGP,

       R – RIP, I L1 – IS-IS level 1, I L2 – IS-IS level 2,

       O3 – OSPFv3, A B – BGP Aggregate, A O – OSPF Summary,

       NG – Nexthop Group Static Route, V – VXLAN Control Service,

       DH – DHCP client installed default route, M – Martian,

       DP – Dynamic Policy Route, L – VRF Leaked


Gateway of last resort is not set


 B E      10.10.21.10/32 [200/0] via VTEP 21.21.21.21 VNI 1000 router-mac 16:3e:9d:e3:fc:e4

 C        10.10.24.0/24 is directly connected, Vlan24

3. VLAN 4094 is the dynamic VLAN that is created on both the NVEs L1 & L4 for the VRF tenant-a:

root@t12_vm8:/home/ece792# docker exec -it L1 Cli -c “show vlan”

VLAN  Name                           Status Ports

—– ——————————– ——— ——————————-

1     default                          active

21    VLAN0021                         active Cpu, Et11, Vx1

4094* VLAN4094                         active Cpu, Vx1


* indicates a Dynamic VLAN

root@t12_vm8:/home/ece792# docker exec -it L4 Cli -c “show vlan”

VLAN  Name                           Status Ports

—– ——————————– ——— ——————————-

1     default                          active

24    VLAN0024                         active Cpu, Et41, Vx1

4094* VLAN4094                         active Cpu, Vx1


* indicates a Dynamic VLAN

4. When an ICMP request is initiated from H1 to H4, the ICMP echo request VXLAN encapsulated by L1 reaches L4:

L1(config)#tcpdump interface ethernet 91 verbose

tcpdump: listening on eth91, link-type EN10MB (Ethernet), capture size 262144 bytes17:08:39.653110 da:16:e8:86:7d:9a > e6:60:9e:50:f3:f9, ethertype IPv4 (0x0800), length 148: (tos 0x0, ttl 64, id 1, offset 0, flags [DF], proto UDP (17), length 134)

    21.21.21.21.46659 > 24.24.24.24.4789: VXLAN, flags [I] (0x08), vni 1000

16:3e:9d:e3:fc:e4 > b6:b4:7a:6b:e3:87, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 63, id 51637, offset 0, flags [DF], proto ICMP (1), length 84)

    10.10.21.10 > 10.10.24.40: ICMP echo request, id 1135, seq 1, length 64

17:08:40.665110 da:16:e8:86:7d:9a > e6:60:9e:50:f3:f9, ethertype IPv4 (0x0800), length 148: (tos 0x0, ttl 64, id 1, offset 0, flags [DF], proto UDP (17), length 134)

    21.21.21.21.46659 > 24.24.24.24.4789: VXLAN, flags [I] (0x08), vni 1000

16:3e:9d:e3:fc:e4 > b6:b4:7a:6b:e3:87, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 63, id 51657, offset 0, flags [DF], proto ICMP (1), length 84)

    10.10.21.10 > 10.10.24.40: ICMP echo request, id 1135, seq 2, length 64

L4(config)#tcpdump interface ethernet 91 ver

tcpdump: listening on eth91, link-type EN10MB (Ethernet), capture size 262144 bytes

17:10:05.657443 86:0a:6a:ec:b5:c1 > fe:02:8c:50:1a:e9, ethertype IPv4 (0x0800), length 148: (tos 0x0, ttl 63, id 1, offset 0, flags [DF], proto UDP (17), length 134)

    21.21.21.21.46659 > 24.24.24.24.4789: VXLAN, flags [I] (0x08), vni 1000

16:3e:9d:e3:fc:e4 > b6:b4:7a:6b:e3:87, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 63, id 61978, offset 0, flags [DF], proto ICMP (1), length 84)

    10.10.21.10 > 10.10.24.40: ICMP echo request, id 1135, seq 85, length 64

5. The VXLAN packet is being decapsulated and sent to the dynamic VLAN:

L4(config)#tcpdump interface vlan 4094 ver

tcpdump: listening on vlan4094, link-type EN10MB (Ethernet), capture size 262144 bytes

^C17:10:59.941997 16:3e:9d:e3:fc:e4 > b6:b4:7a:6b:e3:87, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 63, id 2012, offset 0, flags [DF], proto ICMP (1), length 84)

    10.10.21.10 > 10.10.24.40: ICMP echo request, id 1135, seq 138, length 64

17:11:00.961520 16:3e:9d:e3:fc:e4 > b6:b4:7a:6b:e3:87, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 63, id 2245, offset 0, flags [DF], proto ICMP (1), length 84)

    10.10.21.10 > 10.10.24.40: ICMP echo request, id 1135, seq 139, length 64

6. The packet is not being routed from the dynamic VLAN to VLAN 24. This is where the flow breaks:

L4(config)#tcpdump interface vlan 24

tcpdump: verbose output suppressed, use -v or -vv for full protocol decode

listening on vlan24, link-type EN10MB (Ethernet), capture size 262144 bytes

^C

0 packets captured

0 packets received by filter

0 packets dropped by kernel

L4(config)#

7. The same behavior is observed on L1 when a ping is initiated from H4 towards H1:

L1(config)#

L1(config)#tcpdump interface vlan 4094 verbose

tcpdump: listening on vlan4094, link-type EN10MB (Ethernet), capture size 262144 bytes

^C17:15:02.278953 b6:b4:7a:6b:e3:87 > 16:3e:9d:e3:fc:e4, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 63, id 43196, offset 0, flags [DF], proto ICMP (1), length 84)

    10.10.24.40 > 10.10.21.10: ICMP echo request, id 1042, seq 9, length 64


1 packet captured

1 packet received by filter

0 packets dropped by kernel

L1(config)#tcpdump interface vlan 21 ver

tcpdump: listening on vlan21, link-type EN10MB (Ethernet), capture size 262144 bytes

^C

0 packets captured

0 packets received by filter

0 packets dropped by kernel

L1(config)#

8. Packets are not getting routed from the dynamic VLAN to the VLAN to which the hosts belong. This is the issue being faced here.
9. I tried disabling URPF but it isn’t fixing the issue.

[admin@L1 flash]$ cat > rp_filter.sh
for i in /proc/sys/net/ipv4/conf/*/rp_filter;
do
echo 0 > $i
done
^C
[admin@L1 flash]$ ^C
[admin@L1 flash]$ sudo su
bash-4.3# sh rp_filter.sh
bash-4.3#  cat /proc/sys/net/ipv4/conf/*/rp_filter
0
0
0
0
0
0
0
0
0
0
0
0
0
0
bash-4.3#

[admin@L4 ~]$ sudo su
bash-4.3#
bash-4.3#
bash-4.3# cd /mnt/flash/
bash-4.3# cat > rp_filter.sh
for i in /proc/sys/net/ipv4/conf/*/rp_filter;
do
echo 0 > $i
done
^C
bash-4.3#
bash-4.3# sh rp_filter.sh
bash-4.3# cat /proc/sys/net/ipv4/conf/*/rp_filter
0
0
0
0
0
0
0
0
0
0
0
0
0
0
bash-4.3#
Questions
1. Is this a feature limitation of CEOS wherein it doesn’t support symmetric IRB or am I missing a command here?
2. If it is a feature limitation, is there a later EOS version for Arista switch containers that supports symmetric IRB?
I’ve only able to attach the tech-support files from L1 & L4 because of the size constraint. I can attach the other logs (Qt, syslog, scheduled tech-support and agent logs) in an Egnyte link when shared.
Please let me know if you need anything else.
Regards,
Ashwin
Attachments:
0
Posted by Alexis Dacquay
Answered on March 23, 2020 8:58 am

Hi Ashwin,

I don't see any problem with your configuration.

Can you please provide:

  • show bgp evpn details - from all the devices
  • show run - from the spines

 

I would suggest you upgrade your containers with the latest version, cEOS-lab 4.23.3 at time of writing.

I know it is working with vEOS-lab, but I don't know about cEOS-lab.

Can you try the upgrade and provide the outputs?

 

 

Thanks,

Alexis

 

 

Post your Answer

You must be logged in to post an answer.