Posted on June 8, 2020 5:55 pm
 |  Asked by tomas morales mendoza
 |  48 views
RESOLVED
0
0
Print Friendly, PDF & Email

Hi there

I am trying to build a MPLS SR with EVPN type5 lab using docker-topo. It is a great tool by the way! But I have an issue that I am not sure if it is related to docker-topo, docker, ceos or something else. When checking connectivity inside a VRF (established via EVPN), ping fails. The only difference I can see when checking connectivity in the default VRF is the different source MAC address.

Let me go into the details:

https://github.com/networkop/docker-topo

This is my docker-topo lab:

$ cat 3-node-simple.yml
links:
– endpoints: [“r01:eth0”, “r02:eth0”, “r03:eth0”]
driver: bridge
– endpoints: [“r01:eth1”, “r02:eth1”]
– endpoints: [“r01:eth2”, “r03:eth1”]
– endpoints: [“r02:eth2”, “r03:eth2”]

VERSION: 2
driver: bridge
PREFIX: 3node
CONF_DIR: ./config
CEOS_IMAGE: ceos-lab:4.23.3M
PUBLISH_BASE:
443/tcp: 9000
22/tcp: 2000

These are the interfaces created by docker-topo

$ docker network list
NETWORK ID NAME DRIVER SCOPE
c08d86593b9e 3node_net-0 bridge local
f1e985523447 3node_net-1 bridge local
2d32381310c3 3node_net-2 bridge local
5c55079cda98 3node_net-3 bridge local
6ff18c2afa9e bridge bridge local
c751fb93f368 host host local
9d94e5e7de30 none null local

And this is the detail of net-1 so you can check the MAC addresses involved for r01 and r02:

$ docker network inspect f1e985523447
[
{
“Name”: “3node_net-1”,
“Id”: “f1e985523447276f0d0760abc30c9c36eceaadef6627c831fcc6d649ed2a605a”,
“Created”: “2020-06-06T16:37:34.560808598+01:00”,
“Scope”: “local”,
“Driver”: “bridge”,
“EnableIPv6”: false,
“IPAM”: {
“Driver”: “default”,
“Options”: null,
“Config”: [
{
“Subnet”: “172.19.0.0/16”,
“Gateway”: “172.19.0.1”
}
]
},
“Internal”: false,
“Attachable”: false,
“Ingress”: false,
“ConfigFrom”: {
“Network”: “”
},
“ConfigOnly”: false,
“Containers”: {
“60b16d18d53aba536baafa2290d9e7450840f463dc115fd9c5a105756da0b5cd”: {
“Name”: “3node_r02”,
“EndpointID”: “60b6eddbc0b3dc806311044e9e41a1d9e60eb8b59d26cf44e408abe5fb06b806”,
“MacAddress”: “02:42:ac:13:00:03”,
“IPv4Address”: “172.19.0.3/16”,
“IPv6Address”: “”
},
“f6a3fd71f389c576c0d67d5a8c8cc328bb4c04e4384928041c8fcbbf25d480e7”: {
“Name”: “3node_r01”,
“EndpointID”: “65bd477b31e22044feab9c06bafe9d2aed37ff53557ae0575ae7f4e06fb2ce49”,
“MacAddress”: “02:42:ac:13:00:02”,
“IPv4Address”: “172.19.0.2/16”,
“IPv6Address”: “”
}
},
“Options”: {},
“Labels”: {
“3node”: “3node_net-1”
}
}
]

 

If I ping from r01 to r02 Lo1, it is ok. This doesnt flow via MPLS.

 

r01#ping 10.0.0.2
PING 10.0.0.2 (10.0.0.2) 72(100) bytes of data.
80 bytes from 10.0.0.2: icmp_seq=1 ttl=64 time=0.153 ms
80 bytes from 10.0.0.2: icmp_seq=2 ttl=64 time=0.036 ms
80 bytes from 10.0.0.2: icmp_seq=3 ttl=64 time=0.031 ms
80 bytes from 10.0.0.2: icmp_seq=4 ttl=64 time=0.030 ms
80 bytes from 10.0.0.2: icmp_seq=5 ttl=64 time=0.029 ms

 

And this is what I see if I tcpdump the docker interface

 

$ sudo tcpdump -i br-f1e985523447 -e icmp or mpls -s0 -nnn -vvvvv
tcpdump: listening on br-f1e985523447, link-type EN10MB (Ethernet), capture size 262144 bytes

16:59:43.452350 02:42:ac:13:00:02 > 02:42:ac:13:00:03, ethertype IPv4 (0x0800), length 114: (tos 0x0, ttl 64, id 60795, offset 0, flags [none], proto ICMP (1), length 100)
10.0.12.1 > 10.0.0.2: ICMP echo request, id 2859, seq 1, length 80
16:59:43.452431 02:42:ac:13:00:03 > 02:42:ac:13:00:02, ethertype IPv4 (0x0800), length 114: (tos 0x0, ttl 64, id 39633, offset 0, flags [none], proto ICMP (1), length 100)
10.0.0.2 > 10.0.12.1: ICMP echo reply, id 2859, seq 1, length 80
16:59:43.452485 02:42:ac:13:00:02 > 02:42:ac:13:00:03, ethertype IPv4 (0x0800), length 114: (tos 0x0, ttl 64, id 60796, offset 0, flags [none], proto ICMP (1), length 100)
10.0.12.1 > 10.0.0.2: ICMP echo request, id 2859, seq 2, length 80
16:59:43.452506 02:42:ac:13:00:03 > 02:42:ac:13:00:02, ethertype IPv4 (0x0800), length 114: (tos 0x0, ttl 64, id 39634, offset 0, flags [none], proto ICMP (1), length 100)
10.0.0.2 > 10.0.12.1: ICMP echo reply, id 2859, seq 2, length 80
16:59:43.452593 02:42:ac:13:00:02 > 02:42:ac:13:00:03, ethertype IPv4 (0x0800), length 114: (tos 0x0, ttl 64, id 60797, offset 0, flags [none], proto ICMP (1), length 100)
10.0.12.1 > 10.0.0.2: ICMP echo request, id 2859, seq 3, length 80
16:59:43.452617 02:42:ac:13:00:03 > 02:42:ac:13:00:02, ethertype IPv4 (0x0800), length 114: (tos 0x0, ttl 64, id 39635, offset 0, flags [none], proto ICMP (1), length 100)
10.0.0.2 > 10.0.12.1: ICMP echo reply, id 2859, seq 3, length 80
16:59:43.452687 02:42:ac:13:00:02 > 02:42:ac:13:00:03, ethertype IPv4 (0x0800), length 114: (tos 0x0, ttl 64, id 60798, offset 0, flags [none], proto ICMP (1), length 100)
10.0.12.1 > 10.0.0.2: ICMP echo request, id 2859, seq 4, length 80
16:59:43.452710 02:42:ac:13:00:03 > 02:42:ac:13:00:02, ethertype IPv4 (0x0800), length 114: (tos 0x0, ttl 64, id 39636, offset 0, flags [none], proto ICMP (1), length 100)
10.0.0.2 > 10.0.12.1: ICMP echo reply, id 2859, seq 4, length 80
16:59:43.452783 02:42:ac:13:00:02 > 02:42:ac:13:00:03, ethertype IPv4 (0x0800), length 114: (tos 0x0, ttl 64, id 60799, offset 0, flags [none], proto ICMP (1), length 100)
10.0.12.1 > 10.0.0.2: ICMP echo request, id 2859, seq 5, length 80
16:59:43.452807 02:42:ac:13:00:03 > 02:42:ac:13:00:02, ethertype IPv4 (0x0800), length 114: (tos 0x0, ttl 64, id 39637, offset 0, flags [none], proto ICMP (1), length 100)
10.0.0.2 > 10.0.12.1: ICMP echo reply, id 2859, seq 5, length 80

As you can see in the tcpdump output, the MAC addresses are the ones stated in the docker interface config.

 

if I ping from r01 to r02 Lo2 (in vrf CUST-A) ping fails and the source MAC is different:

 

r01#ping vrf CUST-A 192.168.0.2
PING 192.168.0.2 (192.168.0.2) 72(100) bytes of data.

— 192.168.0.2 ping statistics —
5 packets transmitted, 0 received, 100% packet loss, time 40ms

r01#

 

$ sudo tcpdump -i br-f1e985523447 -e icmp or mpls -s0 -nnn -vvv
tcpdump: listening on br-f1e985523447, link-type EN10MB (Ethernet), capture size 262144 bytes

17:07:27.256275 02:42:ac:f9:ee:11 > 02:42:ac:13:00:03, ethertype MPLS unicast (0x8847), length 118: MPLS (label 116384, exp 0, [S], ttl 65)
(tos 0x0, ttl 65, id 15311, offset 0, flags [none], proto ICMP (1), length 100)
192.168.0.1 > 192.168.0.2: ICMP echo request, id 3187, seq 1, length 80
17:07:27.266857 02:42:ac:f9:ee:11 > 02:42:ac:13:00:03, ethertype MPLS unicast (0x8847), length 118: MPLS (label 116384, exp 0, [S], ttl 65)
(tos 0x0, ttl 65, id 15314, offset 0, flags [none], proto ICMP (1), length 100)
192.168.0.1 > 192.168.0.2: ICMP echo request, id 3187, seq 2, length 80
17:07:27.276862 02:42:ac:f9:ee:11 > 02:42:ac:13:00:03, ethertype MPLS unicast (0x8847), length 118: MPLS (label 116384, exp 0, [S], ttl 65)
(tos 0x0, ttl 65, id 15316, offset 0, flags [none], proto ICMP (1), length 100)
192.168.0.1 > 192.168.0.2: ICMP echo request, id 3187, seq 3, length 80
17:07:27.287324 02:42:ac:f9:ee:11 > 02:42:ac:13:00:03, ethertype MPLS unicast (0x8847), length 118: MPLS (label 116384, exp 0, [S], ttl 65)
(tos 0x0, ttl 65, id 15318, offset 0, flags [none], proto ICMP (1), length 100)
192.168.0.1 > 192.168.0.2: ICMP echo request, id 3187, seq 4, length 80
17:07:27.297519 02:42:ac:f9:ee:11 > 02:42:ac:13:00:03, ethertype MPLS unicast (0x8847), length 118: MPLS (label 116384, exp 0, [S], ttl 65)
(tos 0x0, ttl 65, id 15321, offset 0, flags [none], proto ICMP (1), length 100)
192.168.0.1 > 192.168.0.2: ICMP echo request, id 3187, seq 5, length 80

 

So when the traffic needs to be encapsulated with a MPLS tag, the source MAC address is different. And the source is the one that ceos uses in the egress interface.

 

This is the interfaces config from r1 and r2 (full configs attached)

 

r02#show interfaces e1
Ethernet1 is up, line protocol is up (connected)
Hardware is Ethernet, address is 0242.ac8e.5e00
Internet address is 10.0.12.2/30
Broadcast address is 255.255.255.255
IP MTU 1500 bytes (default)
Full-duplex, Unconfigured, auto negotiation: off, uni-link: n/a
Up 48 minutes, 28 seconds
Loopback Mode : None
2 link status changes since last clear
Last clearing of “show interface” counters never
5 minutes input rate 0 bps (- with framing overhead), 0 packets/sec
5 minutes output rate 0 bps (- with framing overhead), 0 packets/sec
4565 packets input, 904378 bytes
Received 1 broadcasts, 892 multicast
0 runts, 0 giants
0 input errors, 0 CRC, 0 alignment, 0 symbol, 0 input discards
0 PAUSE input
0 packets output, 0 bytes
Sent 0 broadcasts, 0 multicast
0 output errors, 0 collisions
0 late collision, 0 deferred, 0 output discards
0 PAUSE output
r02#
r02#show interfaces e1
Ethernet1 is up, line protocol is up (connected)
Hardware is Ethernet, address is 0242.ac8e.5e00
Internet address is 10.0.12.2/30
Broadcast address is 255.255.255.255
IP MTU 1500 bytes (default)
Full-duplex, Unconfigured, auto negotiation: off, uni-link: n/a
Up 48 minutes, 35 seconds
Loopback Mode : None
2 link status changes since last clear
Last clearing of “show interface” counters never
5 minutes input rate 0 bps (- with framing overhead), 0 packets/sec
5 minutes output rate 0 bps (- with framing overhead), 0 packets/sec
4573 packets input, 905104 bytes
Received 1 broadcasts, 894 multicast
0 runts, 0 giants
0 input errors, 0 CRC, 0 alignment, 0 symbol, 0 input discards
0 PAUSE input
0 packets output, 0 bytes
Sent 0 broadcasts, 0 multicast
0 output errors, 0 collisions
0 late collision, 0 deferred, 0 output discards
0 PAUSE output
r02#show running-config interfaces e1
interface Ethernet1
no switchport
ip address 10.0.12.2/30
isis enable CORE
isis metric 50
isis network point-to-point
r02#
r02#show running-config interfaces loopback 1
interface Loopback1
description CORE Loopback
ip address 10.0.0.2/32
node-segment ipv4 index 2
isis enable CORE
isis metric 1
r02#
r02#show running-config interfaces loopback 2
interface Loopback2
vrf CUST-A
ip address 192.168.0.2/32
r02#
r02#show ip route vrf CUST-A

VRF: CUST-A
Codes: C – connected, S – static, K – kernel,
O – OSPF, IA – OSPF inter area, E1 – OSPF external type 1,
E2 – OSPF external type 2, N1 – OSPF NSSA external type 1,
N2 – OSPF NSSA external type2, B – BGP, B I – iBGP, B E – eBGP,
R – RIP, I L1 – IS-IS level 1, I L2 – IS-IS level 2,
O3 – OSPFv3, A B – BGP Aggregate, A O – OSPF Summary,
NG – Nexthop Group Static Route, V – VXLAN Control Service,
DH – DHCP client installed default route, M – Martian,
DP – Dynamic Policy Route, L – VRF Leaked

Gateway of last resort is not set

B I 192.168.0.1/32 [200/0] via 10.0.0.1/32, IS-IS SR tunnel index 2, label 116384
via 10.0.12.1, Ethernet1, label imp-null(3)
C 192.168.0.2/32 is directly connected, Loopback2
B I 192.168.0.3/32 [200/0] via 10.0.0.3/32, IS-IS SR tunnel index 1, label 116384
via 10.0.23.2, Ethernet2, label imp-null(3)

r02#

 

r01#show interfaces e1
Ethernet1 is up, line protocol is up (connected)
Hardware is Ethernet, address is 0242.acf9.ee11
Internet address is 10.0.12.1/30
Broadcast address is 255.255.255.255
IP MTU 1500 bytes (default)
Full-duplex, Unconfigured, auto negotiation: off, uni-link: n/a
Up 48 minutes, 45 seconds
Loopback Mode : None
2 link status changes since last clear
Last clearing of “show interface” counters never
5 minutes input rate 0 bps (- with framing overhead), 0 packets/sec
5 minutes output rate 0 bps (- with framing overhead), 0 packets/sec
4479 packets input, 896241 bytes
Received 0 broadcasts, 903 multicast
0 runts, 0 giants
0 input errors, 0 CRC, 0 alignment, 0 symbol, 0 input discards
0 PAUSE input
115 packets output, 14030 bytes
Sent 0 broadcasts, 0 multicast
0 output errors, 0 collisions
0 late collision, 0 deferred, 0 output discards
0 PAUSE output
r01#
r01#show running-config interfaces e1
interface Ethernet1
no switchport
ip address 10.0.12.1/30
isis enable CORE
isis metric 50
isis network point-to-point
r01#show running-config interfaces lo1
interface Loopback1
description CORE Loopback
ip address 10.0.0.1/32
node-segment ipv4 index 1
isis enable CORE
isis metric 1
r01#
r01#show running-config interfaces lo2
interface Loopback2
vrf CUST-A
ip address 192.168.0.1/32
r01#
r01#show ip route vrf CUST-A

VRF: CUST-A
Codes: C – connected, S – static, K – kernel,
O – OSPF, IA – OSPF inter area, E1 – OSPF external type 1,
E2 – OSPF external type 2, N1 – OSPF NSSA external type 1,
N2 – OSPF NSSA external type2, B – BGP, B I – iBGP, B E – eBGP,
R – RIP, I L1 – IS-IS level 1, I L2 – IS-IS level 2,
O3 – OSPFv3, A B – BGP Aggregate, A O – OSPF Summary,
NG – Nexthop Group Static Route, V – VXLAN Control Service,
DH – DHCP client installed default route, M – Martian,
DP – Dynamic Policy Route, L – VRF Leaked

Gateway of last resort is not set

C 192.168.0.1/32 is directly connected, Loopback2
B I 192.168.0.2/32 [200/0] via 10.0.0.2/32, IS-IS SR tunnel index 2, label 116384
via 10.0.12.2, Ethernet1, label imp-null(3)
B I 192.168.0.3/32 [200/0] via 10.0.0.3/32, IS-IS SR tunnel index 1, label 116384
via 10.0.13.2, Ethernet2, label imp-null(3)

r01#

 

I can confirm that r02 is receiving always the ICMP traffic (with and without MPLS tag)

 

r02#bash sudo tcpdump -i eth1 icmp or mpls

tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth1, link-type EN10MB (Ethernet), capture size 262144 bytes

16:36:30.287948 02:42:ac:13:00:02 (oui Unknown) > 02:42:ac:13:00:03 (oui Unknown), ethertype IPv4 (0x0800), length 114: 10.0.12.1 > 10.0.0.2: ICMP echo request, id 4307, seq 1, length 80
16:36:30.287991 02:42:ac:13:00:03 (oui Unknown) > 02:42:ac:13:00:02 (oui Unknown), ethertype IPv4 (0x0800), length 114: 10.0.0.2 > 10.0.12.1: ICMP echo reply, id 4307, seq 1, length 80
16:36:30.288839 02:42:ac:13:00:02 (oui Unknown) > 02:42:ac:13:00:03 (oui Unknown), ethertype IPv4 (0x0800), length 114: 10.0.12.1 > 10.0.0.2: ICMP echo request, id 4307, seq 2, length 80
16:36:30.288862 02:42:ac:13:00:03 (oui Unknown) > 02:42:ac:13:00:02 (oui Unknown), ethertype IPv4 (0x0800), length 114: 10.0.0.2 > 10.0.12.1: ICMP echo reply, id 4307, seq 2, length 80
16:36:30.288955 02:42:ac:13:00:02 (oui Unknown) > 02:42:ac:13:00:03 (oui Unknown), ethertype IPv4 (0x0800), length 114: 10.0.12.1 > 10.0.0.2: ICMP echo request, id 4307, seq 3, length 80
16:36:30.288971 02:42:ac:13:00:03 (oui Unknown) > 02:42:ac:13:00:02 (oui Unknown), ethertype IPv4 (0x0800), length 114: 10.0.0.2 > 10.0.12.1: ICMP echo reply, id 4307, seq 3, length 80
16:36:30.289040 02:42:ac:13:00:02 (oui Unknown) > 02:42:ac:13:00:03 (oui Unknown), ethertype IPv4 (0x0800), length 114: 10.0.12.1 > 10.0.0.2: ICMP echo request, id 4307, seq 4, length 80
16:36:30.289053 02:42:ac:13:00:03 (oui Unknown) > 02:42:ac:13:00:02 (oui Unknown), ethertype IPv4 (0x0800), length 114: 10.0.0.2 > 10.0.12.1: ICMP echo reply, id 4307, seq 4, length 80
16:36:30.289120 02:42:ac:13:00:02 (oui Unknown) > 02:42:ac:13:00:03 (oui Unknown), ethertype IPv4 (0x0800), length 114: 10.0.12.1 > 10.0.0.2: ICMP echo request, id 4307, seq 5, length 80
16:36:30.289133 02:42:ac:13:00:03 (oui Unknown) > 02:42:ac:13:00:02 (oui Unknown), ethertype IPv4 (0x0800), length 114: 10.0.0.2 > 10.0.12.1: ICMP echo reply, id 4307, seq 5, length 80

16:36:36.839520 02:42:ac:f9:ee:11 (oui Unknown) > 02:42:ac:13:00:03 (oui Unknown), ethertype MPLS unicast (0x8847), length 118: MPLS (label 116384, exp 0, [S], ttl 65) 192.168.0.1 > 192.168.0.2: ICMP echo request, id 4328, seq 1, length 80
16:36:36.851409 02:42:ac:f9:ee:11 (oui Unknown) > 02:42:ac:13:00:03 (oui Unknown), ethertype MPLS unicast (0x8847), length 118: MPLS (label 116384, exp 0, [S], ttl 65) 192.168.0.1 > 192.168.0.2: ICMP echo request, id 4328, seq 2, length 80
16:36:36.861610 02:42:ac:f9:ee:11 (oui Unknown) > 02:42:ac:13:00:03 (oui Unknown), ethertype MPLS unicast (0x8847), length 118: MPLS (label 116384, exp 0, [S], ttl 65) 192.168.0.1 > 192.168.0.2: ICMP echo request, id 4328, seq 3, length 80
16:36:36.871274 02:42:ac:f9:ee:11 (oui Unknown) > 02:42:ac:13:00:03 (oui Unknown), ethertype MPLS unicast (0x8847), length 118: MPLS (label 116384, exp 0, [S], ttl 65) 192.168.0.1 > 192.168.0.2: ICMP echo request, id 4328, seq 4, length 80
16:36:36.881833 02:42:ac:f9:ee:11 (oui Unknown) > 02:42:ac:13:00:03 (oui Unknown), ethertype MPLS unicast (0x8847), length 118: MPLS (label 116384, exp 0, [S], ttl 65) 192.168.0.1 > 192.168.0.2: ICMP echo request, id 4328, seq 5, length 80

^C
15 packets captured
15 packets received by filter
0 packets dropped by kernel
r02#

So in summary, from r01 perspective, traffic that is not in MPLS uses the docker MAC 02:42:ac:13:00:02, MPLS traffic uses ceos eth1 MAC 02:42:ac:f9:ee:11

Is that expected?  I am assuming that r02 is not answering ICMP when coming from MPLS because it doesnt recognize the source MAC, maybe this is red herring. At the end of the day, I just want to get connectivity across the VRFs.

Thanks

tomas

 

Attachments:
0
Answered on June 10, 2020 11:55 am

cEOS-lab doesn't support MPLS dataplane

Post your Answer

You must be logged in to post an answer.